Skip to content Skip to sidebar Skip to footer
Showing posts with the label Databricks
How To Read Parquet Files Under A Directory Using Pyspark?

How To Read Parquet Files Under A Directory Using Pyspark?

Post a Comment
I have searched online and the solutions provided online didn't resolve my issue. I am trying t… Read more How To Read Parquet Files Under A Directory Using Pyspark?
Not Able To Copy File From Dbfs To Local Desktop In Databricks

Not Able To Copy File From Dbfs To Local Desktop In Databricks

Post a Comment
I want to save or copy my file from the dbfs to my desktop (local). I use this command but get an e… Read more Not Able To Copy File From Dbfs To Local Desktop In Databricks
How To Pass The Script Path To %run Magic Command As A Variable In Databricks Notebook?

How To Pass The Script Path To %run Magic Command As A Variable In Databricks Notebook?

Post a Comment
I want to run a notebook in datarbricks from another notebook using %run. Also I want to be able to… Read more How To Pass The Script Path To %run Magic Command As A Variable In Databricks Notebook?
Perform A User Defined Function On A Column Of A Large Pyspark Dataframe Based On Some Columns Of Another Pyspark Dataframe On Databricks

Perform A User Defined Function On A Column Of A Large Pyspark Dataframe Based On Some Columns Of Another Pyspark Dataframe On Databricks

Post a Comment
My question is relevant to my previous one at How to efficiently join large pyspark dataframes and … Read more Perform A User Defined Function On A Column Of A Large Pyspark Dataframe Based On Some Columns Of Another Pyspark Dataframe On Databricks
Drop Table In Sql Database From Azure Databricks

Drop Table In Sql Database From Azure Databricks

Post a Comment
I need to drop a table in SQL Database using Azure Databricks (PySpark). I am required to not use a… Read more Drop Table In Sql Database From Azure Databricks