Unsuccessful attempts to obtain improvement in ANN regression model results (KERAS)

by Corse   Last Updated February 11, 2019 09:26 AM

I am attempting to build a ANN regression model in KERAS with 3 inputs and 3 outputs (I have attached link to the data I was trying on). Also, I am using Hyperas to optimize the hyperparameters.

My pseudocode is as follows:

read_csv()
StandardScaler(3_inputs)
MinMaxScaler(3_outputs)

def create_model()

4-layers using 4 instances of:    
Dense({{choice([np.power(2,1),np.power(2,2),np.power(2,3)])}}, activation={{choice(['tanh','relu', 'sigmoid'])}})(inputs1)
Dropout({{uniform(0, 1)}})

3-outputs are defined using 3 instances of:
Dense(1,  activation={{choice(['tanh','relu', 'sigmoid'])}})

activation_fn (adam, SGD, rmsprop) are given by:
activation_fn = keras.optimizers.activation_fn (lr={{choice([10**-3,10**-2, 10**-1])}})

model.fit(inputs, outputs,
          batch_size={{choice([16,32,64,128])}},
          epochs={{choice([10000])}},
          verbose=2,
          validation_data=...,
          callbacks=callbacks_list)

I ran the current formulation for the above problem, with number of evaluation runs set at 100. Using either 'mae' or 'mse' as the error, I am getting an actual mean error of 225, 0.25, 45 for each of the outputs respectively.

I have also attempted setting a high number of neurons such as np.power(2,10), however it appears that the error remains stuck at approximately the values above.

Anyone knows if there is any way at all where I can obtain lower errors on the network? I have also tried weighting the losses from each output, and it didn't seem to help. Priority for error reduction is on output 1, then output 3 then output 2.

Appreciate your advice on this, have been hitting the wall on this one. Thank you

Tags : keras


Related Questions


Updated January 14, 2018 08:26 AM

Updated October 18, 2018 05:26 AM

Updated March 07, 2017 04:26 AM

Updated December 06, 2017 08:26 AM

Updated January 15, 2018 20:26 PM