Image by pch.vector in freepik
Machine learning is a broad field in which new research emerges frequently. It’s a hot field where academia and industry keep experimenting with new things to improve our daily lives.
In recent years, generative AI has changed the world through the application of machine learning. For example, ChatGPT and Stable Broadcast. Even with 2023 dominated by generative AI, we must be aware of many more advances in machine learning.
Here are the top machine learning papers to read in 2023 so you don’t miss out on the upcoming trends.
1) Learning the beauty of songs: Neural singing voice beautifier
Singing Voice Beautifying (SVB) is a novel task in generative AI that aims to enhance the amateur singing voice into a beautiful one. It is exactly the research objective of Liu et al. (2022) when they proposed a new generative model called the Neural Singing Voice Beautifier (NSVB).
The NSVB is a semi-supervised learning model that uses a latent mapping algorithm that acts as a pitch corrector and improves vocal pitch. The work promises to improve the music industry and is worth checking out.
2) Symbolic discovery of optimization algorithms
Deep neural network models have become larger than ever and a lot of research has been done to simplify the training process. Recent research from the Google team (Chen et al. (2023)) has proposed a new optimization for the Neural Network called Lion (EvoLved Sign Momentum). The method shows that the algorithm is more memory efficient and requires a lower learning rate than Adam. It’s great research that shows a lot of promise that you shouldn’t miss.
3) TimesNet: 2D temporal variation modeling for general time series analysis
Time series analysis is a common use case in many companies; For example, price forecasting, anomaly detection, etc. However, there are many challenges to analyzing temporal data based only on current data (1D data). That is why Wu et al. (2023) propose a new method called TimesNet to transform 1D data into 2D data, which achieves great performance in the experiment. You should read the paper to better understand this new method as it would help a lot in future time series analysis.
4) OPT: Open Pretrained Transformer Language Models
Today, we are in an era of generative AI in which companies intensively developed many large language models. For the most part, this type of research would not release their model or would only be commercially available. However, the Meta AI research group (Zhang et al. (2022)) tries to do the opposite by publicly releasing the Open Pre-trained Transformers (OPT) model that could be comparable to the GPT-3. The document is a great start to understanding the OPT model and the details of the research, as the group records all the details in the document.
5) REaLTabFormer: generation of realistic relational and tabular data using transformers
The generative model is not limited to only generating text or images, but also tabular data. This generated data is often called synthetic data. Many models were developed to generate synthetic tabular data, but almost no models to generate relational tabular synthetic data. This is exactly the goal of Solatorio and Dupriez (2023) investigation; creating a model called REaLTabFormer for synthetic relational data. The experiment has shown that the result is precisely close to the existing synthetic model, which could be extended to many applications.
6) Is Reinforcement Learning (Not) for Natural Language Processing?: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization
Reinforcement learning is conceptually a great fit for the natural language processing task, but is it true? This is a question that Ramamurthy et al. (2022) try to answer The researcher presents several libraries and algorithms that show where reinforcement learning techniques have an advantage compared to the supervised method in NLP tasks. It is a recommended document to read if you want an alternative for your skill set.
7) Tune-A-Video: One-shot tuning of image diffusion models for text-to-video generation
Text-to-image generation was big in 2022, and 2023 would project into text-to-video (T2V) capability. research by Wu et al. (2022) shows how T2V can be extended in many approaches. The research proposes a new Tune-a-Video method that supports T2V tasks such as subject and object switching, style transfer, attribute editing, etc. It’s a great article to read if you’re interested in text-to-video research.
8) PyGlove: efficient brainstorming of ML as code
Efficient collaboration is the key to success in any team, especially with the increasing complexity within machine learning fields. To promote efficiency, Peng et al. (2023) introduce a PyGlove library to easily share ML ideas. The concept of PyGlove is to capture the ML vetting process through a list of patching rules. The list can then be reused in any experiment scene, improving team efficiency. It’s research trying to solve a machine learning problem that many haven’t done yet, so it’s worth a read.
8) How close is ChatGPT to human experts? Comparison, Evaluation and Detection Corpus
ChatGPT has changed the world a lot. It’s safe to say that the trend would go up from here, as the public is already in favor of using ChatGPT. However, how is the current result of ChatGPT compared to the human experts? It is exactly a question that Guo et al. (2023) try to answer The team tried to collect expert data and quick results from ChatGPT, which they compared. The result shows that there were implicit differences between ChatGPT and the experts. The research is something I think will continue to be asked in the future, as the generative AI model will continue to grow over time, so it’s worth a read.
2023 is a big year for machine learning research showing the current trend, especially generative AI like ChatGPT and Stable Diffusion. There is a lot of promising research that I think we shouldn’t miss because it shows promising results that could change the current standard. In this article, I’ve shown you 9 top ML documents to read, ranging from Generative Model, Time Series Model to Workflow Efficiency. I hope that helps.
Cornelius Yudha Wijaya he is an assistant data science manager and data writer. While working full-time at Allianz Indonesia, she loves to share Python tips and data through social media and writing outlets.
