I have used Hyperband automatic tuning for an ANN model to predict price. After running HyberBand automatic tuning to get the 'best' architecture, I am obtaining a strange Val/Training loss learning curve. I cannot figure out if this is due to an error within the code or just a case of me not understanding the graph and not be able to interpret why the graph is showing as it is. I am also obtaining an R2 score of 1.00 which may suggest overfitting. I've not come across a learning curve (only shown the most basic learning curves at Uni) such as this as of yet so any advice would be greatly appreciated!
Here is the code for the actual tuning, in case it is due to a coding error but I am not sure that is the case.
def model_builder(hp):
model = tf.keras.Sequential()
model.add(tf.keras.layers.Flatten(input_dim = (train_final.shape[1])))
#creating activation choices - choosing betweeen relu and tanh
hp_activation = hp.Choice('activation', values = ['relu', 'tanh'])
#creating node choices - maxing unit amounts to 500
hp_layer_1 = hp.Int('layer_1', min_value=1, max_value=500, step=100)
hp_layer_2 = hp.Int('layer_2', min_value=1, max_value=500, step=100)
#creating learning rate choice - choice between 0.01, 0.001, 0.0001
hp_learning_rate = hp.Choice('learning_rate', values = [1e-2, 1e-3, 1e-4])
#specifies first layer after the flatten layer
model.add(tf.keras.layers.Dense(units = hp_layer_1, activation = hp_activation))
#creating the second layer
model.add(tf.keras.layers.Dense(units = hp_layer_2, activation = hp_activation))
model.add(tf.keras.layers.Dense(1, activation='linear'))
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate = hp_learning_rate),
loss tf.keras.losses.MeanSquaredError(), metrics = ['mean_absolute_error'])
return model
import keras_tuner as kt
#creating the tuner
tuner = kt.Hyperband(model_builder,
objective = 'val_loss',
max_epochs = 50,
factor = 3,
directory = 'dir',
project_name = 'x',
overwrite = True) # makes tuner rewrite over old tuning experiments
#adding early stopping - stops each model from running too long
stop_early = tf.keras.callbacks.EarlyStopping(monitor = 'val_loss', patience = 5)
tuner.search(train_final, y_train, epochs = 50, validation_split = 0.2, callbacks = [stop_early])
best_hp = tuner.get_best_hyperparameters(num_trials=1)[0]
best_hp.values
#obtaining the best model
best_model = tuner.get_best_models(num_models = 1)[0]
history = best_model.fit(train_final, y_train, epochs = 50, validation_split = 0.2, callbacks=[stop_early])
tuned_df = pd.DataFrame(history.history)
#running epoch loss visual def
epoch_loss_visual(tuned_df, model_name = 'Automatic Tuning Model')
Could it be an issue with the code itself causing the issue or is it simply the way the model is? If it's a case of it's just a bad model, I do not need to improve at the moment, but do need to understand the results, especially that of the learning curve representation.
submitted by