Hello, TensorFlow 2#

2.1 Setting Up Your Environment#

Here’s a quick overview of how to set up your environment:

1. Install Python: TensorFlow 2 requires Python 3.7 or above. You can verify your Python version via:

   1
   python --version
   

2. Create a Virtual Environment (optional, but recommended for project isolation):

   1
   python -m venv tf2_env
   2
   source tf2_env/bin/activate  # On Linux/Mac
   3
   tf2_env\Scripts\activate     # On Windows
   

3. Install TensorFlow:

   1
   pip install --upgrade pip
   2
   pip install tensorflow
   

   To install TensorFlow with GPU support (NVIDIA GPUs), ensure you have CUDA and cuDNN configured correctly, and then run:

   1
   pip install tensorflow-gpu
   

   As of TensorFlow 2.x releases, tensorflow-gpu may be included under the main tensorflow package, depending on your platform. Check the official documentation for the latest instructions.

2.2 Verifying the Installation#

To confirm that TensorFlow is correctly installed, open a Python shell or Jupyter notebook and run:

1
import tensorflow as tf
2
print(tf.__version__)

If this prints out a 2.x version (e.g., 2.10.0), congratulations! You have TensorFlow 2 up and running.

3.1 Eager Execution#

Eager execution is now the default mode, meaning TensorFlow operations execute immediately as they are called (rather than building and running a separate computational graph). This immediate feedback is highly beneficial for:

• Interactive Debugging: You can inspect variables on the fly without building a separate session/graph.
• Natural Control Flow: Standard Python constructs (like loops and conditionals) can be used directly for more complexity in your models.

3.2 Tensors#

Tensors are multidimensional arrays. In TensorFlow, tensors flow between operations, forming the basis of all computations. Here’s a quick example of creating and manipulating tensors:

1
import tensorflow as tf
2

3
# Creating Tensors
4
x = tf.constant([[1, 2], [3, 4]], dtype=tf.float32)
5
y = tf.constant([[5, 6], [7, 8]], dtype=tf.float32)
6

7
# Performing Basic Operations
8
z = tf.add(x, y)
9
print(z)  # [[ 6.  8.]
10
          #  [10. 12.]]

3.3 Keras API#

Keras is the high-level neural network API that is now deeply integrated within TensorFlow. It allows for rapid prototyping through a user-friendly, Pythonic interface. Typical usage entails:

• Defining your model using either the Functional API, Sequential API, or Model subclassing.
• Compiling the model, specifying an optimizer, loss function, and metrics.
• Training the model on data with a single .fit(…) call.

4.1 Loading Data#

TensorFlow provides built-in utility functions to load well-known datasets, such as MNIST:

1
import tensorflow as tf
2

3
# Load MNIST dataset
4
mnist = tf.keras.datasets.mnist
5
(x_train, y_train), (x_test, y_test) = mnist.load_data()
6

7
# Reshape and normalize
8
x_train = x_train.reshape(-1, 28, 28, 1).astype("float32") / 255.0
9
x_test  = x_test.reshape(-1, 28, 28, 1).astype("float32") / 255.0

4.2 Creating the Model#

We use the Sequential API for clarity. We will build a simple architecture with convolutional layers, followed by a classification layer:

1
model = tf.keras.Sequential([
2
    tf.keras.layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),
3
    tf.keras.layers.MaxPooling2D((2, 2)),
4
    tf.keras.layers.Conv2D(64, (3, 3), activation='relu'),
5
    tf.keras.layers.MaxPooling2D((2, 2)),
6
    tf.keras.layers.Flatten(),
7
    tf.keras.layers.Dense(64, activation='relu'),
8
    tf.keras.layers.Dense(10, activation='softmax')
9
])

4.3 Compiling and Training#

When compiling the model, you specify an optimizer, a loss function, and metrics to monitor:

1
model.compile(
2
    optimizer='adam',
3
    loss='sparse_categorical_crossentropy',
4
    metrics=['accuracy']
5
)
6

7
# Train the model
8
model.fit(x_train, y_train, epochs=5, batch_size=64)

4.4 Evaluating the Model#

Once the training is complete, evaluate your model on the test set:

1
test_loss, test_acc = model.evaluate(x_test, y_test, verbose=2)
2
print(f"Test accuracy: {test_acc}")

You should see a test accuracy generally above 98% after a few epochs, showcasing TensorFlow’s simplicity in building a straightforward neural network solution.

5.1 Custom Layers#

You can create custom layers by subclassing tf.keras.layers.Layer. This can be powerful when you want more control than what the built-in layers offer:

1
class MyCustomLayer(tf.keras.layers.Layer):
2
    def __init__(self, units=32, **kwargs):
3
        super(MyCustomLayer, self).__init__(**kwargs)
4
        self.units = units
5

6
    def build(self, input_shape):
7
        # Create a trainable weight variable for this layer
8
        self.w = self.add_weight(
9
            shape=(input_shape[-1], self.units),
10
            initializer='random_normal',
11
            trainable=True
12
        )
13
        self.b = self.add_weight(
14
            shape=(self.units,),
15
            initializer='zeros',
16
            trainable=True
17
        )
18

19
    def call(self, inputs):
20
        return tf.matmul(inputs, self.w) + self.b
21

22
# Using the custom layer inside a model
23
custom_model = tf.keras.Sequential([
24
    tf.keras.layers.Flatten(input_shape=(28, 28)),
25
    MyCustomLayer(64),
26
    tf.keras.layers.Activation('relu'),
27
    tf.keras.layers.Dense(10, activation='softmax')
28
])

5.2 Customized Training Loop with GradientTape#

While .fit() offers a high-level interface for training, you can write your own custom loop. This is often indispensable when you need precise control over training steps:

1
# Prepare dataset
2
train_dataset = tf.data.Dataset.from_tensor_slices((x_train, y_train))
3
train_dataset = train_dataset.batch(64)
4

5
# Define model, loss, and optimizer
6
model = tf.keras.Sequential([
7
    tf.keras.layers.Flatten(),
8
    tf.keras.layers.Dense(128, activation="relu"),
9
    tf.keras.layers.Dense(10)
10
])
11
loss_fn = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)
12
optimizer = tf.keras.optimizers.Adam()
13

14
# Custom training loop
15
for epoch in range(5):
16
    for step, (features, labels) in enumerate(train_dataset):
17
        with tf.GradientTape() as tape:
18
            predictions = model(features, training=True)
19
            loss_value = loss_fn(labels, predictions)
20

21
        grads = tape.gradient(loss_value, model.trainable_variables)
22
        optimizer.apply_gradients(zip(grads, model.trainable_variables))
23

24
    print(f"Epoch {epoch+1}: Loss = {loss_value.numpy():.4f}")

This loop demonstrates how you can manually compute gradients and apply them, offering maximum flexibility for specialized tasks.

5.3 Convolutional Neural Networks (CNNs)#

We touched on CNNs briefly in our simple example. CNNs excel at image-related tasks. A typical block includes multiple convolutional layers, each followed by an activation function (e.g., ReLU), and possibly a pooling operation. Dropout layers can be added to reduce overfitting. For more sophisticated tasks such as image segmentation or object detection, you might explore advanced, pre-built models in tf.keras.applications or build your own.

5.4 Saving and Loading Models#

You can save models in two main formats: the HDF5 format and the TensorFlow SavedModel format.

1
# Saving the model as an HDF5 file
2
model.save("my_model.h5")
3

4
# Loading the model
5
new_model = tf.keras.models.load_model("my_model.h5")

For the more flexible and recommended TensorFlow SavedModel format:

1
# Saving
2
model.save("saved_model/")
3

4
# Loading
5
loaded_model = tf.keras.models.load_model("saved_model/")

5.5 Transfer Learning#

Transfer learning allows you to use a pre-trained model (trained on a large dataset) as a starting point. You can freeze the earlier layers (so weights are not updated) and only train the last few layers for your specific dataset. For instance, using a pre-trained MobileNet:

1
base_model = tf.keras.applications.MobileNetV2(
2
    input_shape=(224, 224, 3),
3
    include_top=False,
4
    weights='imagenet'
5
)
6
base_model.trainable = False  # Freeze the base model
7

8
global_avg_layer = tf.keras.layers.GlobalAveragePooling2D()
9
prediction_layer = tf.keras.layers.Dense(10, activation='softmax')
10

11
transfer_model = tf.keras.Sequential([
12
    base_model,
13
    global_avg_layer,
14
    prediction_layer
15
])
16

17
transfer_model.compile(
18
    optimizer='adam',
19
    loss='categorical_crossentropy',
20
    metrics=['accuracy']
21
)

This approach can significantly speed up training and improve accuracy when you have limited data but can leverage a large, established dataset like ImageNet.

7.1 tf.function#

tf.function is a decorator that compiles your Python functions into a static graph, resulting in faster, graph-optimized execution. For instance:

1
@tf.function
2
def compute_predictions(model, x):
3
    return model(x)
4

5
preds = compute_predictions(model, x_test)

Compared to eager mode, graph-based execution runs significantly faster because it can optimize operations and reduce overhead.

7.2 Mixed Precision Training#

Mixed precision training uses both float16 and float32 data types to reduce memory usage and speed up computation on GPUs:

1
# Set global policy
2
from tensorflow.keras import mixed_precision
3
mixed_precision.set_global_policy('mixed_float16')
4

5
# Build and train your model as usual
6
# The weights are stored in float32, while activations use float16

By mixing precision, you can often achieve speedups on GPUs that house specialized TensorCore units for half-precision math (e.g., NVIDIA Volta, Turing, Ampere architectures) while maintaining model accuracy.