Microsoft AI launched Longrope2: an almost loss to extend the context windows of the large language model to 128k tokens while retaining more than 97% precision of short context
Large language models (LLM) have advanced significantly, but a key limitation remains their inability to process long context sequences effectively. ...