A Neptune.ai Guide to Track Your Experiments in Python
Every data scientist is familiar with experimentation.
You already know what to do. You get a data set, load it into a Jupyter notebook, explore it, preprocess the data, tune one or two reference models, and then train an initial final model, such as XGBoost. The first time, maybe don't tune the hyperparameters and include 20 functions. Then, you check your error metrics.
They look good, but maybe their model is a little tight. So you decide to adjust some regularization parameters (e.g. maximum depth) to reduce the complexity of the model and run it again.
You see a small improvement over the last run, but you might also want to:
- Add more features
- Perform feature selection and delete some features.
- Try a different scaler for your functions
- Adjust different/more hyperparameters
As the different types of tests you want to perform increase, the harder it will be to remember which combinations of your “experiments” actually produced the best results. You can only run a notebook so many times, print the results, and copy and paste them into a Google Doc before you get frustrated.
