In MLflow 2.10, we're introducing a number of significant new features that are preparing the way for current and future enhanced support for Deep Learning use cases, new features to support a broadened support for GenAI applications, and some quality of life improvements for the MLflow Deployments Server (formerly the AI Gateway).

New MLflow Website

We have a new home. The new site landing page is fresh, modern, and contains more content than ever. We're adding new content and blogs all of the time.

Model Signature Supports Objects and Arrays (#9936, @serena-ruan)

Objects and Arrays are now available as configurable input and output schema elements. These new types are particularly useful for GenAI-focused flavors that can have complex input and output types. See the new Signature and Input Example documentation to learn more about how to use these new signature types.

Langchain Autologging (#10801, @serena-ruan)

LangChain has autologging support now! When you invoke a chain, with autologging enabled, we will automatically log most chain implementations, recording and storing your configured LLM application for you. See the new Langchain documentation to learn more about how to use this feature.

Prompt Templating for Transformers Models (#10791, @daniellok-db)

The MLflow transformers flavor now supports prompt templates. You can now specify an application-specific set of instructions to submit to your GenAI pipeline in order to simplify, streamline, and integrate sets of system prompts to be supplied with each input request. Check out the updated guide to transformers to learn more and see examples!

MLflow Deployments Server Enhancement (#10765, @gabrielfu; #10779, @TomeHirata)

The MLflow Deployments Server now supports two new requested features: (1) OpenAI endpoints that support streaming responses. You can now configure an endpoint to return realtime responses for Chat and Completions instead of waiting for the entire text contents to be completed. (2) Rate limits can now be set per endpoint in order to help control cost overrun when using SaaS models.

Further Document Improvements

Continued the push for enhanced documentation, guides, tutorials, and examples by expanding on core MLflow functionality (Deployments, Signatures, and Model Dependency management), as well as entirely new pages for GenAI flavors. Check them out today!

Other Features:

• [Models] Enhance the MLflow Models predict API to serve as a pre-logging validator of environment compatibility. (#10759, @B-Step62)
• [Models] Add support for Image Classification pipelines within the transformers flavor (#10538, @KonakanchiSwathi)
• [Models] Add support for retrieving and storing license files for transformers models (#10871, @BenWilson2)
• [Models] Add support for model serialization in the Visual NLP format for JohnSnowLabs flavor (#10603, @C-K-Loan)
• [Models] Automatically convert OpenAI input messages to LangChain chat messages for pyfunc predict (#10758, @dbczumar)
• [Tracking] Enhance async logging functionality by ensuring flush is called on Futures objects (#10715, @chenmoneygithub)
• [Tracking] Add support for a non-interactive mode for the login() API (#10623, @henxing)
• [Scoring] Allow MLflow model serving to support direct dict inputs with the messages key (#10742, @daniellok-db, @B-Step62)
• [Deployments] Add streaming support to the MLflow Deployments Server for OpenAI streaming return compatible routes (#10765, @gabrielfu)
• [Deployments] Add support for directly interfacing with OpenAI via the MLflow Deployments server (#10473, @prithvikannan)
• [UI] Introduce a number of new features for the MLflow UI (#10864, @daniellok-db)
• [Server-infra] Add an environment variable that can disallow HTTP redirects (#10655, @daniellok-db)
• [Artifacts] Add support for Multipart Upload for Azure Blob Storage (#10531, @gabrielfu)

Bug fixes

• [Models] Add deduplication logic for pip requirements and extras handling for MLflow models (#10778, @BenWilson2)
• [Models] Add support for paddle 2.6.0 release (#10757, @WeichenXu123)
• [Tracking] Fix an issue with an incorrect retry default timeout for urllib3 1.x (#10839, @BenWilson2)
• [Recipes] Fix an issue with MLflow Recipes card display format (#10893, @WeichenXu123)
• [Java] Fix an issue with metadata collection when using Streaming Sources on certain versions of Spark where Delta is the source (#10729, @daniellok-db)
• [Scoring] Fix an issue where SageMaker tags were not propagating correctly (#9310, @clarkh-ncino)
• [Windows / Databricks] Fix an issue with executing Databricks run commands from within a Window environment (#10811, @wolpl)
• [Models / Databricks] Disable mlflowdbfs mounts for JohnSnowLabs flavor due to flakiness (#9872, @C-K-Loan)

Documentation updates:

• [Docs] Fixed the KeyError: 'loss' bug for the Quickstart guideline (#10886, @yanmxa)
• [Docs] Relocate and supplement Model Signature and Input Example docs (#10838, @BenWilson2)
• [Docs] Add the HuggingFace Model Evaluation Notebook to the website (#10789, @BenWilson2)
• [Docs] Rewrite the search run documentation (#10863, @chenmoneygithub)
• [Docs] Create documentation for transformers prompt templates (#10836, @daniellok-db)
• [Docs] Refactoring of the Getting Started page (#10798, @BenWilson2)
• [Docs] Add a guide for model dependency management (#10807, @B-Step62)
• [Docs] Add tutorials and guides for LangChain (#10770, @BenWilson2)
• [Docs] Refactor portions of the Deep Learning documentation landing page (#10736, @chenmoneygithub)
• [Docs] Refactor and overhaul the Deployment documentation and add new tutorials (#10726, @B-Step62)
• [Docs] Add a PyTorch landing page, quick start, and guide (#10687, #10737 @chenmoneygithub)
• [Docs] Add additional tutorials to OpenAI flavor docs (#10700, @BenWilson2)
• [Docs] Enhance the guides on quickly getting started with MLflow by demonstrating how to use Databricks Community Edition (#10663, @BenWilson2)
• [Docs] Create the OpenAI Flavor landing page and intro notebooks (#10622, @BenWilson2)
• [Docs] Refactor the Tensorflow flavor API docs (#10662, @chenmoneygithub)

10538, #10901, #10903, #10876, #10833, #10859, #10867, #10843, #10857, #10834, #10814, #10805, #10764, #10771, #10733, #10724, #10703, #10710, #10696, #10691, #10692, @B-Step62; #10882, #10854, #10395, #10725, #10695, #10712, #10707, #10667, #10665, #10654, #10638, #10628, @harupy; #10881, #10875, #10835, #10845, #10844, #10651, #10806, #10786, #10785, #10781, #10741, #10772, #10727, @serena-ruan; #10873, #10755, #10750, #10749, #10619, @WeichenXu123; #10877, @amueller; #10852, @QuentinAmbard; #10822, #10858, @gabrielfu; #10862, @jerrylian-db; #10840, @ernestwong-db; #10841, #10795, #10792, #10774, #10776, #10672, @BenWilson2; #10827, #10826, #10825, #10732, #10481, @michael-berk; #10828, #10680, #10629, @daniellok-db; #10799, #10800, #10578, #10782, #10783, #10723, #10464, @annzhang-db; #10803, #10731, #10708, @kriscon-db; #10797, @dbczumar; #10756, #10751, @Ankit8848; #10784, @AveshCSingh; #10769, #10763, #10717, @chenmoneygithub; #10698, @rmalani-db; #10767, @liangz1; #10682, @cdreetz; #10659, @prithvikannan; #10639, #10609, @TomeHirata