#databrickssql

[hubertdudek]

Databricks: what’s new in May 2025? Updates & Features Explained! #databricks Databricks, What’s New in Databricks? May 2025 Updates & Features Explained! In May 2025, Databricks added several key features. 📌 Key Highlights for This Month: - *0:16* 16.4 LTS - *0:28* Autoloader auto cleaner - *2:28* Lakeflow UI connectors - *3:01* Workflow run with different settings - *4:27* ETL/DLT editor - *5:30* PRIVATE materialised views and streaming tables - *6:48* Delta share materialised views and streaming tables - *7:27* Clean rooms up to 10 collaborators - *7:57* Predictive optimisation for all - *8:45* Just-in-time user provisioning - *10:04* Cluster logs - *11:13* Run the code inside the assistant - *13:22* Query snippets - *14:34* New charts - *15:43* Run apps locally - *16:51* Custom data sources - *18:01* Syntax highlighter - *19:25* String aggregation ============================= 📚 *Notebooks from the video:* 🔗 [GitHub Repository](https://ift.tt/aJpTNju) 🔔𝐃𝐨𝐧'𝐭 𝐟𝐨𝐫𝐠𝐞𝐭 𝐭𝐨 𝐬𝐮𝐛𝐬𝐜𝐫𝐢𝐛𝐞 𝐭𝐨 𝐦𝐲 𝐜𝐡𝐚𝐧𝐧𝐞𝐥 𝐟𝐨𝐫 𝐦𝐨𝐫𝐞 𝐮𝐩𝐝𝐚𝐭𝐞𝐬. https://www.youtube.com/@databricks_hubert_dudek/?sub_confirmation=1 🔗 Support Me Here! ☕Buy me a coffee: https://ift.tt/nlEDgNR ✨ Explore Databricks AI insights and workflows—read more: https://ift.tt/hUeGRFE ============================= 🎬Suggested videos for you: ▶️ [What’s new in January 2025](https://www.youtube.com/watch?v=JJiwSplZmfk) ▶️ [What’s new in February 2025](https://www.youtube.com/watch?v=tuKI0sBNbmg) ▶️ [What’s new in March 2025](https://youtu.be/hJD7KoNq-uE) ▶️ [What’s new in April 2025](https://youtu.be/FDgtNVeLTc8) ============================= 📚 **New Articles for Further Reading:** - 📝 *Clean Landing Zone — autoloader cleanSource:* 🔗 [Read the full article](https://ift.tt/gS2h1s3) - 📝 *Nested groups in databricks:* 🔗 [Read the full article](https://ift.tt/TileUHn) - 📝 *Cost Benchmark: 2 billion records from bronze to silver on serverless:* 🔗 [Read the full article](https://ift.tt/WUnICfR) - 📝 *Logs to Volumes and to Dataframe:* 🔗 [Read the full article](https://ift.tt/Reya0pJ) ============================= 🔎 Related Phrases: #databricks #bigdata #dataengineering #machinelearning #sql #cloudcomputing #dataanalytics #ai #azure #googlecloud #aws #etl #python #data #database #datawarehouse via databricks by Hubert Dudek https://www.youtube.com/channel/UCR99H9eib5MOHEhapg4kkaQ May 19, 2025 at 03:07AM

[hubertdudek]

Databricks: what’s new in April 2025? Updates & Features Explained! #databricks Databricks, What’s New in Databricks? April 2025 Updates & Features Explained! 📌 Key Highlights for This Month: - *00:04* PowerBI task - Refresh PowerBI from Databricks - *01:36* SQL task values - Pass SELECT result to workflow - *05:38* Cost-optimized jobs - Serverless standard mode - *06:34* Google Sheets - Query Databricks - *07:48* Git for dashboards - *08:38* Genie sampling - Genie can read data - *11:22* UC functions with PyPl libraries - *12:22* Anomaly detection - *15:02* PII scanner - Data classification - *16:13* Turn off Hive metastore - *17:17* AI builder - Extract data and more - *21:12* AI query with schema - *22:41* PyDABS - *23:28* ALTER statement - *24:03* TEMP VIEWS in DLT - *24:18* Apps on behalf of the user ============================= 📚 *Notebooks from the video:* 🔗 [GitHub Repository](https://ift.tt/S13qG0b) 🔔𝐃𝐨𝐧'𝐭 𝐟𝐨𝐫𝐠𝐞𝐭 𝐭𝐨 𝐬𝐮𝐛𝐬𝐜𝐫𝐢𝐛𝐞 𝐭𝐨 𝐦𝐲 𝐜𝐡𝐚𝐧𝐧𝐞𝐥 𝐟𝐨𝐫 𝐦𝐨𝐫𝐞 𝐮𝐩𝐝𝐚𝐭𝐞𝐬. https://www.youtube.com/@hubert_dudek/?sub_confirmation=1 🔗 Support Me Here! ☕Buy me a coffee: https://ift.tt/9qIpuET ✨ Explore Databricks AI insights and workflows—read more: https://ift.tt/1djZykN ============================= 🎬Suggested videos for you: ▶️ [What’s new in January 2025](https://www.youtube.com/watch?v=JJiwSplZmfk) ▶️ [What’s new in February 2025](https://www.youtube.com/watch?v=tuKI0sBNbmg) ▶️ [What’s new in March 2025](https://youtu.be/hJD7KoNq-uE) ============================= 📚 **New Articles for Further Reading:** - 📝 *More on Databricks into Google Sheets:* 🔗 [Read the full article](https://ift.tt/3cfjJLy) - 📝 *More on Anomaly Detection & Data Freshness:* 🔗 [Read the full article](https://ift.tt/5RB4bWM) - 📝 *More on Goodbye to Hive Metastore:* 🔗 [Read the full article](https://ift.tt/lxjpoRS) - 📝 *More on Databricks Refresh PowerBI Semantic Model:* 🔗 [Read the full article](https://ift.tt/8JAfSvZ) - 📝 *More on ResponseFormat in AI Batch Inference:* 🔗 [Read the full article](https://ift.tt/B07yqRT) ============================= 🔎 Related Phrases: #databricks #bigdata #dataengineering #machinelearning #sql #cloudcomputing #dataanalytics #ai #azure #googlecloud #aws #etl #python #data #database #datawarehouse via Hubert Dudek https://www.youtube.com/channel/UCR99H9eib5MOHEhapg4kkaQ April 22, 2025 at 02:17AM

[hubertdudek]

Hubert and Youssef: about Databricks Workflows Databricks Workflows - we talk about improvements and best practices. #databricks #SQL #python #workflows ✨ Explore Databricks AI insights and workflows—read more: https://ift.tt/hWrQOsf 🔔𝐃𝐨𝐧'𝐭 𝐟𝐨𝐫𝐠𝐞𝐭 𝐭𝐨 𝐬𝐮𝐛𝐬𝐜𝐫𝐢𝐛𝐞 𝐭𝐨 𝐦𝐲 𝐜𝐡𝐚𝐧𝐧𝐞𝐥 𝐟𝐨𝐫 𝐦𝐨𝐫𝐞 𝐮𝐩𝐝𝐚𝐭𝐞𝐬. https://www.youtube.com/@hubert_dudek/?sub_confirmation=1 🔗 Support Me Here! ☕Buy me a coffee: https://ift.tt/xE1NpMb 🔗 Stay Connected With Me. Medium: https://ift.tt/hWrQOsf ============================= 🎬Suggested videos for you: ▶️ https://www.youtube.com/watch?v=hJD7KoNq-uE ▶️ https://www.youtube.com/watch?v=JJiwSplZmfk&t=1s ================================= 🔎 Related Phrases: #databricks #bigdata #dataengineering #machinelearning #sql #cloudcomputing #dataanalytics #ai #azure #googlecloud #aws #etl #python #data #database #datawarehouse #python #workflows via Hubert Dudek https://www.youtube.com/channel/UCR99H9eib5MOHEhapg4kkaQ March 22, 2025 at 12:31AM

[hubertdudek]

What’s New in Databricks? March 2025 Updates & Features Explained! ### *🚀 What’s New in Databricks? March 2025 Updates & Features Explained!* #databricks #spark #dataengineering #ai #sql #llm Stay ahead with the *latest Databricks updates* for *March 2025.* This month introduces powerful features like: *SQL scripting enhancements, Calling Agents, Genie Files, Lakeflow, Streaming from Views, Secure Access Tokens, Binds, JSON Metadata Exploration, and Automatic Liquid Clustering.* 📌 *Watch the complete breakdown and see how these updates impact your data workflows!* ✨ *🔍 Key Highlights in This Update:* - *0:10* – SQL Scripting Enhancements: More advanced scripting with `BEGIN...END`, `CASE`, and control flow structures - *0:58* – Tabs: Soft tabs for notebooks and files have landed - *1:38* – MLFlow Trae UI: Debug agents with improved tracking - *2:27* – Calling Agents in Databricks: Connect Databricks to external services (e.g., Jira) using *http_request()* function - *5:50* – Volume File Previews: Seamlessly *preview files in volumes* - *6:15* – Genie Files: Easily *join files in Genie conversations* - *7:57* – Genie REST API: Develop your own app using *out-of-the-box Genie capabilities* - *9:15* – Lakeflow Enhancements: New ingestion pipelines, including *Workday & ServiceNow integrations* - *10:40* – Streaming from Views: Learn how to *stream data from SQL views* into live data pipelines - *11:45* – Secure Access Tokens: Manage Databricks *API tokens securely* - *12:24* – Binds: Improve workspace management with *Databricks workspace bindings* for external locations and credentials - *14:22* – DESCRIBE AS JSON: Explore metadata *directly in JSON format* for *more straightforward automation* - *15:50* – Automatic Liquid Clustering: Boost *query performance* with predictive clustering 📚 *Notebooks from the video:* 🔗 [GitHub Repository](https://ift.tt/c3dZYQh) 📝 *More on SQL Enhancements:* 🔗 [Read the full article](https://ift.tt/n9VX6dq) 📝 *More on DESCRIBE AS JSON:* 🔗 [Read the full article](https://ift.tt/sRPU3ik) 📝 *More on Calling GENIE API:* 🔗 [Read the full article](https://ift.tt/6D5fJrQ) ☕ *Enjoyed the video? Could you support me with a coffee?:* 🔗 [Buy Me a Coffee](https://ift.tt/Xv9AmPY) 💡 Whether you're a *data engineer, analyst, or Databricks enthusiast,* these updates will *enhance your workflows* and boost productivity! 🔔 *Subscribe for more Databricks insights & updates:* 📢 [YouTube Channel](https://www.youtube.com/@hubert_dudek/?sub_confirmation=1) 📢 *Stay Connected:* 🔗 [Medium Blog](https://ift.tt/cpeVd0J) --- ### 🎬 *Recommended Videos:* ▶️ [What’s new in January 2025](https://www.youtube.com/watch?v=JJiwSplZmfk)\ ▶️ [What’s new in February 2025](https://www.youtube.com/watch?v=tuKI0sBNbmg) --- ### *🔎 Related Phrases & Keywords:* What’s New In Databricks, March 2025 Updates, Databricks Latest Features, SQL Scripting in Databricks, Calling Agents with HTTP, Genie File Previews, Lakeflow Pipelines, Streaming from Views, Databricks Access Tokens, Databricks Binds, Metadata in JSON, Automatic Liquid Clustering \#databricks #bigdata #dataengineering #machinelearning #sql #cloudcomputing #dataanalytics #ai #azure #googlecloud #aws #etl #python #data #database #datawarehouse via Hubert Dudek https://www.youtube.com/channel/UCR99H9eib5MOHEhapg4kkaQ March 16, 2025 at 09:55PM