Google Research introduces an extension to PyGlove: a general-purpose Python library that can now easily and scalably share machine learning insights as code

In recent years, machine learning (ML) partnerships have grown significantly in scale, making it more difficult to share code efficiently. Multiple academics and engineers can connect through universities, GitHub projects, and tech companies. There are often many separate teams that share a code base, particularly in technology businesses. Other groups should incorporate the ideas made by these teams into their code. However, challenges can arise due to specialized teams and code bases. The most typical approach is for each team to keep an eye on the findings made by other teams, and then apply those findings to their ML system. This process can take a long time when there are too many innovations or when they are difficult or need specialized knowledge.

There are more difficulties with the alternative technique, such as inadequate access or documentation, when innovators implement their discoveries in other code bases directly. Most significantly, these expenses are spent every time there is a trade between two teams. The same idea is executed more than once, which results in poor scalability as there are more inventions. They present PyGlove in this article as an expansion of their previous work to make it easy to brainstorm at scale as code. A novel idea can be used in multiple places with little implementation work, thanks to PyGlove. By programmatically announcing their discovery, innovators themselves can update other teams’ code. At a high level, PyGlove uses rule-based annotations and corrections.

A code base must first be peppered with properly structured, lightweight Python annotations that explain the code at an understandable level to make it compatible with PyGlove. The shared code will be carried out using annotations as a common language. After completion, the code can be transferred via rule-based patches that specify where the ported code should be. Consider a scenario where “Team A” maintains a classifier for images and “Team B” independently develops a new convolutional layer that should improve most classifiers. According to the PyGlove method, Team A can annotate their (pre-existing) code with phrases like “this is a convolution”, “this is a non-linearity”, etc. Instead, Team B would annotate their new layer with phrases like “these are hyperparameters.”

Team A can create a one-line rule that says, “replace all my convolutions with team B’s layer” after learning about the new layer, as illustrated in Figure 2. A single spin is also possible thanks to PyGlove: team B can create its own replacement rule, which is equivalent to saying, “in each image classifier, replace all convolutions with its layer”. After that, the rule developed by Team B can be used by any team with a PyGlove annotated image classifier. This unexpected turn brings many possibilities for future cooperation through the ML innovation repositories that you describe in your paper. They point out that the convolution layer swap scenario was used as an example because it is relatively basic.

The rule-based approach that PyGlove uses extends to all parts of the ML pipeline, including data augmentation, training algorithms, and meta-learning, and is not just limited to sharing architectural mods. In particular, there is often a need to expand model capability as ML technology advances. To solve this problem, empirical and theoretical principles have been developed. Such regulations can be made known to everyone in a group or neighborhood, saving critical time for the engineer. Due to a “network effect” between teams, the cost of adopting PyGlove can quickly be offset by its advantages. The work required to annotate a code base when only the new annotations need coding is the cost of adoption. Since these are common Python annotations, most of the original code remains intact.

On the other hand, PyGlove offers advantages that teams can take advantage of whenever they share ideas. When m innovations are applied to team projects without PyGlove, the work is min; however, with PyGlove, each innovation requires the creation of a PyGlove rule (m rules), and each team project is in charge of adding PyGlove annotations to their model (n models), resulting in only m + n work anymore. that the application of the rule is trivial. In each of these cases, his rule-based methodology differs from existing approaches, which often require numerous adjustments in place and need to better scale with the size of the model or the number of practitioners in the community.

The open source PyGlove library and supplementary code are used in this document and the open source PyGlove library. For example, their case study of a sizeable code base revealed that the adoption of PyGlove resulted in an 80% decrease in the number of lines of code. Due to the fundamental symbolic programming nature of PyGlove, it can be used to write ML code in all its facets and code for other purposes outside of ML. This paradigm transforms Python objects annotated with PyGlove into editable symbols, and PyGlove rules are metaprograms that operate on these symbols. In summary, they present a method for effectively and scalably sharing complex ML ideas as code using symbolic patches. An example of how symbolic programming can be used throughout the ML development process. PyGlove is open source and instructions for use can be found on its GitHub.

review the Paper and Github. All credit for this research goes to the researchers of this project. Also, don’t forget to join our 13k+ ML SubReddit, discord channel, and electronic newsletterwhere we share the latest AI research news, exciting AI projects, and more.