New Features

• Add dynamic task mapping (https://github.com/apache/airflow/pulls?q=is%3Apr+is%3Amerged+label%3AAIP-42+milestone%3A%22Airflow+2.3.0%22)
• New Grid View replaces Tree View (#18675)
• Templated requirements.txt in Python Operators (#17349)
• Allow reuse of decorated tasks (#22941)
• Move the database configuration to a new section (#22284)
• Add SmoothOperator (#22813)
• Make operator's execution_timeout configurable (#22389)
• Events Timetable (#22332)
• Support dag serialization with custom ti_deps rules (#22698)
• Support log download in task log view (#22804)
• support for continue backfill on failures (#22697)
• Add dag-processor cli command (#22305)
• Add possibility to create users in LDAP mode (#22619)
• Add ignore_first_depends_on_past for scheduled jobs (#22491)
• Update base sensor operator to support XCOM return value (#20656)
• Add an option for run id in the ui trigger screen (#21851)
• Enable JSON serialization for connections (#19857)
• Add REST API endpoint for bulk update of DAGs (#19758)
• Add queue button to click-on-DagRun interface. (#21555)
• Add list-import-errors to airflow dags command (#22084)
• Store callbacks in database if standalone_dag_processor config is True. (#21731)
• Add LocalKubernetesExecutor (#19729)
• Add celery.task_timeout_error metric (#21602)
• Airflow db downgrade cli command (#21596)
• Add ALL_SKIPPED trigger rule (#21662)
• Add db clean CLI command for purging old data (#20838)
• Add celery_logging_level (#21506)
• Support different timeout value for dag file parsing (#21501)
• Support generating SQL script for upgrades (#20962)
• Add option to compress Serialized dag data (#21332)
• Branch python operator decorator (#20860)
• Add Audit Log View to Dag View (#20733)
• Add missing StatsD metric for failing SLA Callback notification (#20924)
• Add ShortCircuitOperator configurability for respecting downstream trigger rules (#20044)
• Allow using Markup in page title in Webserver (#20888)
• Add Listener Plugin API that tracks TaskInstance state changes (#20443)
• Add context var hook to inject more env vars (#20361)
• Add a button to set all tasks to skipped (#20455)
• Cleanup pending pods (#20438)
• Add config to warn public deployment exposure in UI (#18557)
• Log filename template records (#20165)
• Added windows extensions (#16110)
• Showing approximate time until next dag_run in Airflow (#20273)
• Extend config window on UI (#20052)
• Add show dag dependencies feature to CLI (#19985)
• Add cli command for 'airflow dags reserialize` (#19471)
• Add missing description field to Pool schema(REST API) (#19841)
• Introduce DagRun action to change state to queued. (#19353)
• Add DAG run details page (#19705)
• Add role export/import to cli tools (#18916)
• Adding dag_id_pattern parameter to the /dags endpoint (#18924)

Improvements

• Show schedule_interval/timetable description in UI (#16931)
• Added column duration to DAG runs view (#19482)
• Enable use of custom conn extra fields without prefix (#22607)
• Initialize finished counter at zero (#23080)
• Improve logging of optional provider features messages (#23037)
• Meaningful error message in resolve_template_files (#23027)
• Update ImportError items instead of deleting and recreating them (#22928)
• Add option --skip-init to db reset command (#22989)
• Support importing connections from files with ".yml" extension (#22872)
• Support glob syntax in .airflowignore files (#21392) (#22051)
• Hide pagination when data is a single page (#22963)
• Support for sorting DAGs in the web UI (#22671)
• Speed up has_access decorator by ~200ms (#22858)
• Add XComArg to lazy-imported list of Airflow module (#22862)
• Add more fields to REST API dags/dag_id/details endpoint (#22756)
• Don't show irrelevant/duplicated/"internal" Task attrs in UI (#22812)
• No need to load whole ti in current_state (#22764)
• Pickle dag exception string fix (#22760)
• Better verification of Localexecutor's parallelism option (#22711)
• log backfill exceptions to sentry (#22704)
• retry commit on MySQL deadlocks during backfill (#22696)
• Add more fields to REST API get DAG(dags/dag_id) endpoint (#22637)
• Use timetable to generate planned days for current year (#22055)
• Disable connection pool for celery worker (#22493)
• Make date picker label visible in trigger dag view (#22379)
• Expose try_number in airflow vars (#22297)
• Add generic connection type (#22310)
• Add a few more fields to the taskinstance finished log message (#22262)
• Pause auto-refresh if scheduler isn't running (#22151)
• Show DagModel details. (#21868)
• Add pip_install_options to PythonVirtualenvOperator (#22158)
• Show import error for airflow dags list CLI command (#21991)
• Pause auto-refresh when page is hidden (#21904)
• Default args type check (#21809)
• Enhance magic methods on XComArg for UX (#21882)
• py files don't have to be checked is_zipfiles in refresh_dag (#21926)
• Fix TaskDecorator type hints (#21881)
• Add 'Show record' option for variables (#21342)
• Use DB where possible for quicker airflow dag subcommands (#21793)
• REST API: add rendered fields in task instance. (#21741)
• Change the default auth backend to session (#21640)
• Don't check if py DAG files are zipped during parsing (#21538)
• Switch XCom implementation to use run_id (#20975)
• Action log on Browse Views (#21569)
• Implement multiple API auth backends (#21472)
• Change logging level details of connection info in get_connection() (#21162)
• Support mssql in airflow db shell (#21511)
• Support config worker_enable_remote_control for celery (#21507)
• Log memory usage in CgroupTaskRunner (#21481)
• Modernize DAG-related URL routes and rename "tree" to "grid" (#20730)
• Move Zombie detection to SchedulerJob (#21181)
• Improve speed to run airflow by 6x (#21438)
• Add more SQL template fields renderers (#21237)
• Simplify fab has access lookup (#19294)
• Log context only for default method (#21244)
• Log trigger status only if at least one is running (#21191)
• Add optional features in providers. (#21074)
• Better multiple_outputs inferral for @task.python (#20800)
• Improve handling of string type and non-attribute template_fields (#21054)
• Remove un-needed deps/version requirements (#20979)
• Correctly specify overloads for TaskFlow API for type-hinting (#20933)
• Introduce notification_sent to SlaMiss view (#20923)
• Rewrite the task decorator as a composition (#20868)
• Add "Greater/Smaller than or Equal" to filters in the browse views (#20602) (#20798)
• Rewrite DAG run retrieval in task command (#20737)
• Speed up creation of DagRun for large DAGs (5k+ tasks) by 25-130% (#20722)
• Make native environment Airflow-flavored like sandbox (#20704)
• Better error when param value has unexpected type (#20648)
• Add filter by state in DagRun REST API (List Dag Runs) (#20485)
• Prevent exponential memory growth in Tasks with custom logging handler (#20541)
• Set default logger in logging Mixin (#20355)
• Reduce deprecation warnings from www (#20378)
• Add hour and minute to time format on x-axis of all charts using nvd3.lineChart (#20002)
• Add specific warning when Task asks for more slots than pool defined with (#20178)
• UI: Update duration column for better human readability (#20112)
• Use Viewer role as example public role (#19215)
• Properly implement DAG param dict copying (#20216)
• ShortCircuitOperator push XCom by returning python_callable result (#20071)
• Add clear logging to tasks killed due to a Dagrun timeout (#19950)
• Change log level for Zombie detection messages (#20204)
• Better confirmation prompts (#20183)
• Only execute TIs of running DagRuns (#20182)
• Check and run migration in commands if necessary (#18439)
• Log only when Zombies exists (#20118)
• Increase length of the email and username (#19932)
• Add more filtering options for TI's in the UI (#19910)
• Dynamically enable "Test Connection" button by connection type (#19792)
• Avoid littering postgres server logs with "could not obtain lock" with HA schedulers (#19842)
• Renamed Connection.get_hook parameter to make it the same as in SqlSensor and SqlOperator. (#19849)
• Add hook_params in SqlSensor using the latest changes from PR #18718. (#18431)
• Speed up webserver boot time by delaying provider initialization (#19709)
• Configurable logging of XCOM value in PythonOperator (#19378)
• Minimize production js files (#19658)
• Add hook_params in BaseSqlOperator (#18718)
• Add missing "end_date" to hash components (#19281)
• More friendly output of the airflow plugins command + add timetables (#19298)
• Add sensor default timeout config (#19119)
• Update taskinstance REST API schema to include dag_run_id field (#19105)
• Adding feature in bash operator to append the user defined env variable to system env variable (#18944)
• Duplicate Connection: Added logic to query if a connection id exists before creating one (#18161)

Bug Fixes

• Use inherited 'trigger_tasks' method (#23016)
• In DAG dependency detector, use class type instead of class name (#21706)
• Fix tasks being wrongly skipped by schedule_after_task_execution (#23181)
• Fix X-Frame enabled behaviour (#23222)
• Allow extra to be nullable in connection payload as per schema(REST API). (#23183)
• Fix dag_id extraction for dag level access checks in web ui (#23015)
• Fix timezone display for logs on UI (#23075)
• Include message in graph errors (#23021)
• Change trigger dropdown left position (#23013)
• Don't add planned tasks for legacy DAG runs (#23007)
• Add dangling rows check for TaskInstance references (#22924)
• Validate the input params in connection CLI command (#22688)
• Fix trigger event payload is not persisted in db (#22944)
• Drop "airflow moved" tables in command db reset (#22990)
• Add max width to task group tooltips (#22978)
• Add template support for external_task_ids. (#22809)
• Allow DagParam to hold falsy values (#22964)
• Fix regression in pool metrics (#22939)
• Priority order tasks even when using pools (#22483)
• Do not clear XCom when resuming from deferral (#22932)
• Handle invalid JSON metadata in get_logs_with_metadata endpoint. (#22898)
• Fix pre-upgrade check for rows dangling w.r.t. dag_run (#22850)
• Fixed backfill interference with scheduler (#22701)
• Support conf param override for backfill runs (#22837)
• Correctly interpolate pool name in PoolSlotsAvailableDep statues (#22807)
• Fix email_on_failure with render_template_as_native_obj (#22770)
• Fix processor cleanup on DagFileProcessorManager (#22685)
• Prevent meta name clash for task instances (#22783)
• remove json parse for gantt chart (#22780)
• Check for missing dagrun should know version (#22752)
• Fixes ScheduleInterval spec (#22635)
• Fixing task status for non-running and non-committed tasks (#22410)
• Do not log the hook connection details even at DEBUG level (#22627)
• Stop crashing when empty logs are received from kubernetes client (#22566)
• Fix bugs about timezone change (#22525)
• Fix entire DAG stops when one task has end_date (#20920)
• Use logger to print message during task execution. (#22488)
• Make sure finalizers are not skipped during exception handling (#22475)
• update smart sensor docs and minor fix on is_smart_sensor_compatible() (#22386)
• Fix run_id k8s and elasticsearch compatibility with Airflow 2.1 (#22385)
• Allow to except_skip None on BranchPythonOperator (#20411)
• Fix incorrect datetime details (DagRun views) (#21357)
• Remove incorrect deprecation warning in secrets backend (#22326)
• Remove RefreshConfiguration workaround for K8s token refreshing (#20759)
• Masking extras in GET /connections/<connection> endpoint (#22227)
• Set queued_dttm when submitting task to directly to executor (#22259)
• Addressed some issues in the tutorial mentioned in discussion #22233 (#22236)
• Change default python executable to python3 for docker decorator (#21973)
• Don't validate that Params are JSON when NOTSET (#22000)
• Add per-DAG delete permissions (#21938)
• Fix handling some None parameters in kubernetes 23 libs. (#21905)
• Fix handling of empty (None) tags in bulk_write_to_db (#21757)
• Fix DAG date range bug (#20507)
• Removed request.referrer from views.py (#21751)
• Make DbApiHook use get_uri from Connection (#21764)
• Fix some migrations (#21670)
• [de]serialize resources on task correctly (#21445)
• Add params dag_id, task_id etc to XCom.serialize_value (#19505)
• Update test connection functionality to use custom form fields (#21330)
• fix all "high" npm vulnerabilities (#21526)
• Fix bug incorrectly removing action from role, rather than permission. (#21483)
• Fix relationship join bug in FAB/SecurityManager with SQLA 1.4 (#21296)
• Use Identity instead of Sequence in SQLAlchemy 1.4 for MSSQL (#21238)
• Ensure on_task_instance_running listener can get at task (#21157)
• Return to the same place when triggering a DAG (#20955)
• Fix task ID deduplication in @task_group (#20870)
• Add downgrade to some FAB migrations (#20874)
• Only validate Params when DAG is triggered (#20802)
• Fix airflow trigger cli (#20781)
• Fix task instances iteration in a pool to prevent blocking (#20816)
• Allow depending to a @task_group as a whole (#20671)
• Use original task's start_date if a task continues after deferral (#20062)
• Disabled edit button in task instances list view page (#20659)
• Fix a package name import error (#20519) (#20519)
• Remove execution_date label when get cleanup pods list (#20417)
• Remove unneeded FAB REST API endpoints (#20487)
• Fix parsing of Cloudwatch log group arn containing slashes (#14667) (#19700)
• Sanity check for MySQL's TIMESTAMP column (#19821)
• Allow using default celery command group with executors subclassed from Celery-based executors. (#18189)
• Move class_permission_name to mixin so it applies to all classes (#18749)
• Adjust trimmed_pod_id and replace '.' with '-' (#19036)
• Pass custom_headers to send_email and send_email_smtp (#19009)
• Ensure catchup=False is used in example dags (#19396)
• Edit permalinks in OpenApi description file (#19244)
• Navigate directly to DAG when selecting from search typeahead list (#18991)
• [Minor] Fix padding on home page (#19025)

Doc only changes

• Update doc for DAG file processing (#23209)
• Replace changelog/updating with release notes and towncrier now (#22003)
• Fix wrong reference in tracking-user-activity.rst (#22745)
• Remove references to rbac = True from docs (#22725)
• Doc: Update description for executor-bound dependencies (#22601)
• Update check-health.rst (#22372)
• Stronger language about Docker Compose customizability (#22304)
• Update logging-tasks.rst (#22116)
• Add example config of sql_alchemy_connect_args (#22045)
• Update best-practices.rst (#22053)
• Add information on DAG pausing/deactivation/deletion (#22025)
• Add brief examples of integration test dags you might want (#22009)
• Run inclusive language check on CHANGELOG (#21980)
• Add detailed email docs for Sendgrid (#21958)
• Add docs for db upgrade / db downgrade (#21879)
• Update modules_management.rst (#21889)
• Fix UPDATING section on SqlAlchemy 1.4 scheme changes (#21887)
• Update TaskFlow tutorial doc to show how to pass "operator-level" args. (#21446)
• Fix doc - replace decreasing by increasing (#21805)
• Add another way to dynamically generate DAGs to docs (#21297)
• Add extra information about time synchronization needed (#21685)
• Update debug.rst docs (#21246)
• Replaces the usage of postgres:// with postgresql:// (#21205)
• Fix task execution process in CeleryExecutor docs (#20783)

Misc/Internal

• Bring back deprecated security manager functions (#23243)
• Replace usage of DummyOperator with EmptyOperator (#22974)
• Deprecate DummyOperator in favor of EmptyOperator (#22832)
• Remove unnecessary python 3.6 conditionals (#20549)
• Bump moment from 2.29.1 to 2.29.2 in /airflow/www (#22873)
• Bump prismjs from 1.26.0 to 1.27.0 in /airflow/www (#22823)
• Bump nanoid from 3.1.23 to 3.3.2 in /airflow/www (#22803)
• Bump minimist from 1.2.5 to 1.2.6 in /airflow/www (#22798)
• Remove dag parsing from db init command (#22531)
• Update our approach for executor-bound dependencies (#22573)
• Use Airflow.Base.metadata in FAB models (#22353)
• Limit docutils to make our documentation pretty again (#22420)
• Add Python 3.10 support (#22050)
• [FEATURE] add 1.22 1.23 K8S support (#21902)
• Remove pandas upper limit now that SQLA is 1.4+ (#22162)
• Patch sql_alchemy_conn if old postgres scheme used (#22333)
• Protect against accidental misuse of XCom.get_value() (#22244)
• Order filenames for migrations (#22168)
• Don't try to auto generate migrations for Celery tables (#22120)
• Require SQLAlchemy 1.4 (#22114)
• bump sphinx-jinja (#22101)
• Add compat shim for SQLAlchemy to avoid warnings (#21959)
• Rename xcom.dagrun_id to xcom.dag_run_id (#21806)
• Deprecate non-JSON conn.extra (#21816)
• Bump upper bound version of jsonschema to 5.0 (#21712)
• Deprecate helper utility days_ago (#21653)
• Remove :type lines now sphinx-autoapi supports type hints (#20951)
• Silence deprecation warning in tests (#20900)
• Use DagRun.run_id instead of execution_date when updating state of TIs (UI & REST API) (#18724)
• Add Context stub to Airflow packages (#20817)
• Update Kubernetes library version (#18797)
• Rename PodLauncher to PodManager (#20576)
• Removes Python 3.6 support (#20467)
• Add deprecation warning for non-json-serializable params (#20174)
• Rename TaskMixin to DependencyMixin (#20297)
• Deprecate passing execution_date to XCom methods (#19825)
• Remove get_readable_dags and get_editable_dags, and get_accessible_dags. (#19961)
• Remove postgres 9.6 support (#19987)
• Removed hardcoded connection types. Check if hook is instance of DbApiHook. (#19639)
• add kubernetes 1.21 support (#19557)
• Add FAB base class and set import_name explicitly. (#19667)
• Removes unused state transitions to handle auto-changing view permissions. (#19153)
• Chore: Use enum for __var and __type members (#19303)
• Use fab models (#19121)
• Consolidate method names between Airflow Security Manager and FAB default (#18726)
• Remove distutils usages for Python 3.10 (#19064)
• Removing redundant max_tis_per_query initialisation on SchedulerJob (#19020)
• Remove deprecated usage of init_role() from API (#18820)
• Remove duplicate code on dbapi hook (#18821)