In a previous article, I wrote about using the timeit magic command to measure the time efficiency of different operations. In this article, I will write about the run magic command and explain how it has helped me to organise my notebooks.

While working on data transformations in Azure Synapse, my notebook became unwieldy due to its length. To enhance clarity and usability, I opted to break it down into multiple files. Sharing variables across these files streamlined the process and improved overall organization.

I started researching how best to go about this and came across the run magic command.

---

What is the %run Magic Command?

This magic command is a line magic command and helps to run Python scripts or notebooks as a program in another notebook namespace.

For better understanding, I will explain this use case: I am performing a data transformation task across different datasets which share the same constants. To avoid rewriting these constants in each notebook, I decided to create a separate file for defining these constants. This file would be called Constants and the notebooks would be called Notebook1 and Notebook2.

To use this file containing constants across each notebook, I can use this syntax: run <filename>

The code snippets below illustrate the use case above using code:

Constants.py

Date = 'Test_Date'
Time = 'Test_Time'

Notebook1.py

%run Constants

# let's assume a dataframe variable 'df' already exists
# and we want to create a new column with the Date and Time constants

df = df.withColumn(Date, '2023-03-10')
df = df.withColumn(Time, '03:00:23')

Notebook2.py

%run Constants

# let's assume a dataframe variable 'df' already exists
# and we want to create a new column with the Date and Time constants

df = df.withColumn(Date, '2023-03-11')
df = df.withColumn(Time, '04:00:23')

---

Based on the illustration above, I have been able to utilize the %run magic command to reuse files across my notebooks. This is great because it avoids repetition and simplifies code in the notebooks.

I hope this helps you! 😄