Python for Data Science 101: The Beginner’s Guide#

Jupyter Notebooks#

A large number of data scientists love using Jupyter Notebooks or JupyterLab, which come with Anaconda by default. Jupyter Notebooks allow you to write and run Python code in your browser, interspersed with explanatory text, plots, and other media—perfect for data exploration and storytelling. To start a Jupyter Notebook, simply open a terminal and run:

1
jupyter notebook

Variables and Data Types#

In Python, variables are created as soon as you assign a value to them, and there is no need to declare their type explicitly. Common data types include:

• int: Integer numbers (e.g., 5, 10, -3).
• float: Decimal numbers (e.g., 3.14, 2.718).
• str: Strings (e.g., "Hello").
• bool: Boolean (True or False).

Example:

1
my_int = 10
2
my_float = 3.14
3
my_str = "Hello Data Science!"
4
my_bool = True
5

6
print(type(my_int))   # <class 'int'>
7
print(type(my_float)) # <class 'float'>
8
print(type(my_str))   # <class 'str'>
9
print(type(my_bool))  # <class 'bool'>

Conditional Statements#

Conditional statements (if-elif-else) allow you to execute different blocks of code based on conditions:

1
x = 10
2
if x > 0:
3
    print("x is positive")
4
elif x == 0:
5
    print("x is zero")
6
else:
7
    print("x is negative")

Loops#

Two main loop types in Python are for and while:

1
# For loop
2
for i in range(5):
3
    print(i)
4

5
# While loop
6
count = 0
7
while count < 3:
8
    print("Count:", count)
9
    count += 1

These basics form the foundation of Python. If you’re new, spend some time getting comfortable with them before diving deeper into data science libraries.

Lists#

A list is an ordered collection of items enclosed in square brackets. Lists are mutable, meaning you can change, add, or remove elements:

1
my_list = [1, 2, 3, 4, 5]
2
my_list.append(6)
3
print(my_list)  # [1, 2, 3, 4, 5, 6]
4
my_list[0] = 10
5
print(my_list)  # [10, 2, 3, 4, 5, 6]

Tuples#

Tuples are like lists but are immutable (cannot be changed once created). They are defined using parentheses:

1
my_tuple = (1, 2, 3)
2
# my_tuple[0] = 10  # This will throw a TypeError
3
print(my_tuple)  # (1, 2, 3)

Dictionaries#

Dictionaries store key-value pairs and are unordered. They are incredibly useful for data retrieval by unique identifiers (keys):

1
my_dict = {"name": "Alice", "age": 30, "city": "New York"}
2
print(my_dict["name"])  # Alice
3
my_dict["age"] = 31
4
my_dict["position"] = "Data Scientist"
5
print(my_dict)
6
# {'name': 'Alice', 'age': 31, 'city': 'New York', 'position': 'Data Scientist'}

Sets#

Sets are unordered, unindexed collections used for membership testing and eliminating duplicates:

1
my_set = {1, 2, 3, 3, 2}
2
print(my_set)  # {1, 2, 3}

List Comprehensions#

A powerful Python feature is list comprehensions, which allow you to create lists in a concise way:

1
squares = [x**2 for x in range(5)]
2
print(squares)  # [0, 1, 4, 9, 16]

NumPy#

NumPy provides support for multi-dimensional arrays and a suite of mathematical functions to operate on these arrays efficiently. Its core data structure is the NumPy array, which is similar to a Python list but optimized for numerical computations.

1
import numpy as np
2

3
arr = np.array([1, 2, 3, 4, 5])
4
print(arr.mean())  # 3.0

pandas#

pandas is built on top of NumPy and introduces two main data structures: Series and DataFrame. A DataFrame is analogous to a spreadsheet or SQL table. It’s the go-to for data wrangling and manipulation:

1
import pandas as pd
2

3
data = {
4
    "Name": ["Alice", "Bob", "Charlie"],
5
    "Age": [25, 30, 35],
6
    "City": ["New York", "Chicago", "Los Angeles"]
7
}
8
df = pd.DataFrame(data)
9
print(df)

Matplotlib#

Matplotlib is a foundational plotting library. It allows you to generate a variety of static, animated, and interactive visualizations:

1
import matplotlib.pyplot as plt
2

3
x = [1, 2, 3, 4, 5]
4
y = [2, 4, 1, 8, 7]
5
plt.plot(x, y)
6
plt.xlabel("X-axis Label")
7
plt.ylabel("Y-axis Label")
8
plt.title("Simple Line Plot")
9
plt.show()

seaborn#

seaborn is built on top of Matplotlib and provides a high-level interface for drawing attractive statistical graphics:

1
import seaborn as sns
2

3
# Sample data
4
tips = sns.load_dataset("tips")
5
sns.scatterplot(data=tips, x="total_bill", y="tip", hue="time")
6
plt.show()

scikit-learn#

scikit-learn brings machine learning algorithms like linear regression, decision trees, clustering, and more to your fingertips. It focuses on model building and evaluation:

1
from sklearn.linear_model import LinearRegression
2

3
model = LinearRegression()
4
# Assume you have X (features) and y (target)
5
model.fit(X, y)
6
y_pred = model.predict(X)

Importing Data#

You can import CSV, Excel, or even SQL data using pandas:

1
df_csv = pd.read_csv("data.csv")
2
df_excel = pd.read_excel("data.xlsx")

Inspecting Data#

Use methods such as head(), shape, info(), and describe():

1
print(df.head())
2
print(df.shape)
3
print(df.info())
4
print(df.describe())

Handling Missing Data#

Missing data can hamper analysis. pandas has tools to drop or fill them:

1
df.dropna(inplace=True)                # Drop rows containing NaN values
2
df.fillna(value={"Age": df["Age"].mean()}, inplace=True)  # Fill NaNs with mean

Filtering and Selecting#

You can subset DataFrames using boolean indexing or the loc/iloc operators:

1
filtered_df = df[df["Age"] > 30]
2
print(filtered_df)
3

4
# loc uses labels, iloc uses integer indexes
5
row_0_loc = df.loc[0, "Name"]
6
row_0_iloc = df.iloc[0, 0]

Grouping and Aggregation#

Data analysis often involves summarizing information. Use groupby along with aggregate functions:

1
grouped = df.groupby("City").agg({"Age": "mean"})
2
print(grouped)

Descriptive Statistics#

pandas offers quick methods to compute statistical metrics:

1
print(df["Age"].mean())
2
print(df["Salary"].median())
3
print(df["Salary"].std())

Correlation#

Studying correlations among different variables can help you spot meaningful relationships:

1
correlation_matrix = df.corr()
2
print(correlation_matrix)
3
sns.heatmap(correlation_matrix, annot=True)
4
plt.show()

Outlier Detection#

Box plots and scatter plots are commonly used for outlier detection:

1
sns.boxplot(data=df, x="Age")
2
plt.title("Box Plot of Age")
3
plt.show()

Common Plot Types#

1. Line Plot – Best for continuous data or trends over time.
2. Bar Plot – Compare categories or track changes over time with discrete intervals.
3. Histogram – Display data distribution by grouping values into bins.
4. Box Plot – Highlight distributions and potential outliers.
5. Scatter Plot – Explore relationships between two (or more) variables.

Example:

1
sns.histplot(data=df, x="Age", kde=True)
2
plt.title("Age Distribution")
3
plt.show()

You may also combine multiple plots or use subplots to compare different variables in a single figure.

Basic ML Workflow with scikit-learn#

scikit-learn has a user-friendly and consistent API:

1. Import the model you want to use.
2. Instantiate the model with desired parameters.
3. Fit the model to your training data.
4. Predict outcomes for new or test data.
5. Evaluate the model’s performance.

1
from sklearn.linear_model import LinearRegression
2
from sklearn.model_selection import train_test_split
3
from sklearn.metrics import mean_squared_error
4

5
# Sample data
6
X = df[["YearsExperience"]]  # Features
7
y = df["Salary"]             # Target
8

9
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
10

11
model = LinearRegression()
12
model.fit(X_train, y_train)
13

14
predictions = model.predict(X_test)
15
mse = mean_squared_error(y_test, predictions)
16
print("Mean Squared Error:", mse)

Feature Engineering#

Transforming raw data into more meaningful features can significantly improve model performance. Typical transformations include:

• Normalization or Standardization: Scale numerical data to a uniform range or distribution.
• One-Hot Encoding: Convert categorical variables into dummy variables.
• Binning: Convert continuous variables into intervals or bins.
• Polynomial Features: Capture interactions between features.

1
from sklearn.preprocessing import StandardScaler, OneHotEncoder
2

3
scaler = StandardScaler()
4
X_scaled = scaler.fit_transform(X)
5

6
ohe = OneHotEncoder()
7
categorical_data = ohe.fit_transform(df[["City"]])

Model Selection and Hyperparameter Tuning#

Use techniques like grid search or randomized search to find the best combination of hyperparameters. scikit-learn offers easy methods:

1
from sklearn.model_selection import GridSearchCV
2

3
param_grid = {"fit_intercept": [True, False]}
4
lr_gs = GridSearchCV(LinearRegression(), param_grid, cv=5)
5
lr_gs.fit(X_train, y_train)
6
print(lr_gs.best_params_)
7
print(lr_gs.best_score_)

Deep Learning Frameworks#

For more complex tasks like image classification, natural language processing, or large-scale recommendations, deep learning frameworks are now the gold standard:

• TensorFlow: Developed by Google, suitable for large-scale projects and distributed computing.
• PyTorch: Favored for research and fast prototyping, developed by Facebook’s AI Research lab.

Simple PyTorch example:

1
import torch
2
import torch.nn as nn
3
import torch.optim as optim
4

5
# Define a simple neural network
6
class SimpleNN(nn.Module):
7
    def __init__(self):
8
        super(SimpleNN, self).__init__()
9
        self.fc = nn.Linear(10, 1)
10

11
    def forward(self, x):
12
        return self.fc(x)
13

14
model = SimpleNN()
15
criterion = nn.MSELoss()
16
optimizer = optim.SGD(model.parameters(), lr=0.01)
17

18
# Dummy data
19
inputs = torch.randn(5, 10)
20
targets = torch.randn(5, 1)
21

22
# Training loop (simplified)
23
for epoch in range(50):
24
    optimizer.zero_grad()
25
    outputs = model(inputs)
26
    loss = criterion(outputs, targets)
27
    loss.backward()
28
    optimizer.step()

Deployment#

After successfully training and evaluating your model, you may need to deploy it so others can use it. Common approaches:

• Flask or FastAPI: Expose your model as a REST API endpoint.
• Docker: Containerize your entire environment for consistent deployment across different servers.
• Cloud Platforms: Services like AWS Sagemaker, Google Cloud AI Platform, or Azure Machine Learning for serverless scaling.