[Lesson 1] Introduction to Machine Learning (2)

Machine Learning Model

- This was really impressive considering that we only needed a few lines code:

1) l0 = tf.keras.layers.Dense(units=1, input_shape=[1])

2) model = tf.keras.Sequential([l0])

3) model.compile(loss='mean_squared_error', optimizer=tf.keras.optimizers.Adam(0.1))

4) history = model.fit(celsius_q, fahrenheit_a, epochs=500, verbose=False)

5) model.predict([100.0])

 

- This example is the general plan for of any machine learning program. You will use the same structure to create and train your neural network

 

- This is achieved through an optimization process called Gradient Descent(기울기 하강), which  iteratively adjusts parameters, nudging them in the correct direction a bit at a time until they reach the best values.

 

- The function that measures how good or bad the model is during each iteration is called the “loss function”, and the goal of each nudge is to “minimize the loss function.”

 

- The training process starts with a forward pass, where the input data is fed to the neural network.

FORWARD PASS

- Once a value is predicted, the difference between that predicted value and the correct value is calculated. This difference is called the loss, and it's a measure of how well the model performed the mapping task.

 

- After the loss is calculated, the internal variables (weights and biases) of all the layers of the neural network are adjusted, so as to minimize this loss.

Backpropagation

- This optimization process is called Gradient Descent. learning more details about how the training process works, you can look at the lesson on reducing loss in Google’s machine learning crash course

손실 줄이기  |  머신러닝 단기집중과정  |  Google Developers

 

<Arrangement of Terms>

• Feature: The input(s) to our model
• Examples: An input/output pair used for training
• Labels: The output of the model
• Layer: A collection of nodes connected together within a neural network.
• Model: The representation of your neural network
• Dense and Fully Connected (FC): Each node in one layer is connected to each node in the previous layer.
• Weights and biases: The internal variables of model
• Loss: The discrepancy between the desired output and the actual output
• MSE: Mean squared error, a type of loss function that counts a small number of large discrepancies as worse than a large number of small ones.
• Gradient Descent: An algorithm that changes the internal variables a bit at a time to gradually reduce the loss function.
• Optimizer: A specific implementation of the gradient descent algorithm. (There are many algorithms for this. In this course we will only use the “Adam” Optimizer, which stands for ADAptive with Momentum. It is considered the best-practice optimizer.)
• Learning rate: The “step size” for loss improvement during gradient descent.
• Batch: The set of examples used during training of the neural network
• Epoch: A full pass over the entire training dataset
• Forward pass: The computation of output values from input
• Backward pass (backpropagation): The calculation of internal variable adjustments according to the optimizer algorithm, starting from the output layer and working back through each layer to the input.

- In order to understand what a dense layer is, let's create a slightly more complicated neural network that has three inputs, one hidden layer with two units, and an output layer with only a single unit.

 

- Recall, that I can think of a neural network as a stack of layers, where each layer is made up of units(=neurons).

Dense layer

- every neuron in each layer is connected to all the neurons in the previous layer.

 

- For example, to create there both neural networking keras, we simply have to use the following statements.

hidden = keras.layers.Dense(units=2, input_shape=[3])

output = keras.layers.Dense(units=1)

model = tf.keras.Sequential( [ hidden, output ] )

 

- What happens in the training process is that these weights and biases are being tuned to the best possible values to match the inputs to the outputs.

- What's important to note however, is that the math never changes. the training process only changes the w and b variables to be able to match the input to the output.

 

- In our example converting Celsius to Faherenheit, with the single neuron, we only have one weight w11 and b1(bias) available to tune.

 

- Without knowing the target algorithm, we would just give a model a number of layers and weights to tune. Then, we just hope the model will be able to figure out how to tune the model algorithm to match the input to the output.

 

- In general, when we do machine learning, we typically just try different neural networks with different numbers of layers and neurons in a trial and error away.

 

<Lesson Summary>

• Learn some of the basic concepts behind Machine Learning and how Dense layers work.
• Train my first machine learning model to convert Celsius to Fahrenheit.
• Also learn some of the key terms used in Machine Learning such as features, examples, and labels.
• Also learn the main lines of code that make up the skeleton of any machine learning algorithm.
• See that I only need a few lines of code to create, train, and make predictions with my neural network using TensorFlow and Keras