v0.11.0

• Added databricks labs ucx alias command to create a view of tables from one schema/catalog in another schema/catalog (#837).
• Added databricks labs ucx save-aws-iam-profiles command to scan instance profiles identify AWS S3 access and save a CSV with permissions (#817).
• Added total view counts in the assessment dashboard (#834).
• Cleaned up assess_jobs and assess_clusters tasks in the assessment workflow to improve testing and reduce redundancy.(#825).
• Added documentation for the assessment report (#806).
• Fixed escaping for SQL object names (#836).

Dependency updates:

• Updated databricks-sdk requirement from ~=0.17.0 to ~=0.18.0 (#832).

Contributors: @FastLee, @nfx, @dependabot[bot], @prajin-29, @dmoore247, @larsgeorge-db

v0.10.0

• Added databricks labs ucx validate-groups-membership command to validate groups to see if they have same membership across acount and workspace level (#772).
• Added baseline for getting Azure Resource Role Assignments (#764).
• Added issue and pull request templates (#791).
• Added linked issues to PR template (#793).
• Added optional debug_truncate_bytes parameter to the config and extend the default log truncation limit (#782).
• Added support for crawling grants and applying Hive Metastore UDF ACLs (#812).
• Changed Python requirement from 3.10.6 to 3.10 (#805).
• Extend error handling of delta issues in crawlers and hive metastore (#795).
• Fixed databricks labs ucx repair-run command to execute correctly (#801).
• Fixed handling of DELTASHARING table format (#802).
• Fixed listing of workflows via CLI (#811).
• Fixed logger import path for DEBUG notebook (#792).
• Fixed move table command to delete table/view regardless if permissions are present, skipping corrupted tables when crawling table size and making existing tests more stable (#777).
• Fixed the issue of databricks labs ucx installations and databricks labs ucx manual-workspace-info (#814).
• Increase the unit test coverage for cli.py (#800).
• Mount Point crawler lists /Volume with four variations which is confusing (#779).
• Updated README.md to remove mention of deprecated install.sh (#781).
• Updated bug issue template (#797).
• Fixed writing log readme in multiprocess safe way (#794).

Contributors: @mwojtyczka, @nfx, @larsgeorge-db, @prajin-29, @ggwiebe, @HariGS-DB, @dmoore247, @dipankarkush-db, @qziyuan

v0.9.0

• Added assessment step to estimate the size of DBFS root tables (#741).
• Added TableMapping functionality to table migrate (#752).
• Added databricks labs ucx move command to move tables and schemas between catalogs (#756).
• Added functionality to determine migration method based on DBFS Root (#759).
• Added get_tables_to_migrate functionality in the mapping module (#755).
• Added retry and rate limit to rename workspace group operation and corrected rate limit for reflecting account groups to workspace (#751).
• Adopted databricks-labs-blueprint library for common utilities to be reused in the other projects (#758).
• Converted RuntimeBackend query executions exceptions to SDK exceptions (#769).
• Fixed issue with missing users and temp groups after workspace-local groups migration and skip table when crawling table size if it does not exist anymore (#770).
• Improved error handling by not failing group rename step if a group was removed from account before reflecting it to workspace (#762).
• Improved error message inference from failed workflow runs (#753).
• Moved TablesMigrate to a separate module (#747).
• Reorganized assessment dashboard to increase readability (#738).
• Updated databricks-sdk requirement from ~=0.16.0 to ~=0.17.0 (#773).
• Verify metastore exists in current workspace (#735).

Contributors: @FastLee, @mwojtyczka, @pritishpai, @nfx, @dmoore247, @HariGS-DB

v0.8.0

• Added databricks labs ucx repair-run --step ... CLI command for repair run of any failed workflows, like assessment, migrate-groups etc. (#724).
• Added databricks labs ucx revert-migrated-table command (#729).
• Allow specifying a group list when group match options are used (#725).
• Fixed installation issue when upgrading from an older version of the tool and improve logs (#740).
• Renamed summary panel from Failure Summary to Assessment Summary (#733).
• Retry internal error when getting permissions and update legacy table ACL documentation (#728).
• Speedup installer execution (#727).

Contributors: @nfx, @mwojtyczka, @FastLee, @dmoore247, @HariGS-DB, @prajin-29

v0.7.0

• Added databricks labs ucx create-table-mapping and databricks labs ucx manual-workspace-info commands for CLI (#682).
• Added databricks labs ucx ensure-assessment-run to CLI commands (#708).
• Added databricks labs ucx installations command (#679).
• Added databricks labs ucx skip --schema ... --table ... command to mark table/schema for skipping in the table migration process (#680).
• Added databricks labs ucx validate-external-locations command for cli (#715).
• Added capturing ANY FILE and ANONYMOUS FUNCTION grants (#653).
• Added cluster override and handle case of write protected DBFS (#610).
• Added cluster policy selector in the installer (#655).
• Added detailed UCX pre-requisite information to README.md (#689).
• Added interactive wizard for databricks labs uninstall ucx command (#657).
• Added more granular error retry logic (#704).
• Added parallel fetching of registered model identifiers to speed-up assessment workflow (#691).
• Added retry on workspace listing (#659).
• Added support for mapping workspace group to account group by prefix/suffix/regex/external id (#650).
• Changed cluster security mode from NONE to LEGACY_SINGLE_USER, as crawl_tables was failing when run on non-UC Workspace in No Isolation mode with unable to access the config file (#661).
• Changed the fields of the table "Tables" to lower case (#684).
• Enabled integration tests for EXTERNAL table migrations (#677).
• Enforced mypy validation (#713).
• Filtering out inventory database from loading into tables and filtering out the same from grant detail view (#705).
• Fixed documentation for reflect_account_groups_on_workspace task and updated CONTRIBUTING.md guide (#654).
• Fixed secret scope apply task to raise ValueError (#683).
• Fixed legacy table ACL ownership migration and other integration testing issues (#722).
• Fixed some flaky integration tests (#700).
• New CLI command for workspace mapping (#678).
• Reduce server load for getting workspace groups and their members (#666).
• Throwing ManyError on migrate-groups tasks (#710).
• Updated installation documentation to use Databricks CLI (#686).

Dependency updates:

• Updated databricks-sdk requirement from ~=0.13.0 to ~=0.14.0 (#651).
• Updated databricks-sdk requirement from ~=0.14.0 to ~=0.15.0 (#687).
• Updated databricks-sdk requirement from ~=0.15.0 to ~=0.16.0 (#712).

Contributors: @nfx, @HariGS-DB, @prajin-29, @william-conti, @FastLee, @pritishpai, @dmoore247, @andrascsillag-db, @dipankarkush-db, @mirhendi

v0.6.2

• Added current version of UCX to task logs (#566).
• Fixed 'str' object has no attribute 'value' failure on apply backup group permissions task (#574).
• Fixed crawl_cluster failure over custom runtimes (#602).
• Fixed databricks labs ucx workflows command (#608).
• Fixed problematic integration test fixture make_ucx_group (#613).
• Fixed internal API request retry logic by relying on concrete exception types (#637).
• Fixed tables.scala notebook to read inventory database from ~/.ucx/config.yml file. (#614).
• Introduced StaticTablesCrawler for integration tests (#632).
• Reduced runtime of test_set_owner_permission from 15 minutes to 44 seconds (#636).
• Updated LICENSE (#643).
• Updated documentation (#611, #646).

Breaking changes (existing installations need to remove ucx database, reinstall UCX and re-run assessment jobs)

• Fixed external locations widget to return hostname for jdbc:-sourced tables (#621).

v0.6.1

• Added a logo for UCX (#605).
• Check if the hatch is already installed, and install only if it isn't installed yet (#603).
• Fixed installation check for git pre-release versions (#600).
• Temporarily remove SQL warehouse requirement from labs.yml (#604).

v0.6.0

Breaking changes (existing installations need to reinstall UCX and re-run assessment jobs)

• Switched local group migration component to rename groups instead of creating backup groups (#450).
• Mitigate permissions loss in Table ACLs by folding grants belonging to the same principal, object id and object type together (#512).

New features

• Added support for the experimental Databricks CLI launcher (#517).
• Added support for external Hive Metastores including AWS Glue (#400).
• Added more views to assessment dashboard (#474).
• Added rate limit for creating backup group to increase stability (#500).
• Added deduplication for mount point list (#569).
• Added documentation to describe interaction with external Hive Metastores (#473).
• Added failure injection for job failure message propagation (#591).
• Added uniqueness in the new warehouse name to avoid conflicts on installation (#542).
• Added a global init script to collect Hive Metastore lineage (#513).
• Added retry set/update permissions when possible and assess the changes in the workspace (#519).
• Use ~/.ucx/state.json to store the state of both dashboards and jobs (#561).

Bug fixes

• Fixed handling for OWN table permissions (#571).
• Fixed handling of keys with and without values. (#514).
• Fixed integration test failures related to concurrent group delete (#584).
• Fixed issue with workspace listing process on None type object_type (#481).
• Fixed missing group entitlement migration bug (#583).
• Fixed entitlement application for account-level groups (#529).
• Fixed assessment throwing an error when the owner of an object is empty (#485).
• Fixed installer to migrate between different configuration file versions (#596).
• Fixed cluster policy crawler to be aware of deleted policies (#486).
• Improved error message for not null constraints violated (#532).
• Improved integration test resiliency (#597, #594, #586).
• Introduced Safer access to workspace objects' properties. (#530).
• Mitigated permissions loss in Table ACLs by running appliers with single thread (#518).
• Running apply permission task before assessment should display message (#487).
• Split integration tests from blocking the merge queue (#496).
• Support more than one dashboard per step (#472).
• Update databricks-sdk requirement from ~=0.11.0 to ~=0.12.0 (#505).
• Update databricks-sdk requirement from ~=0.12.0 to ~=0.13.0 (#575).

v0.5.0

• Added make install-dev and a stronger make clean for easier dev on-boarding and release upgrades (#458).
• Added failure summary in the assessment dashboard (#455).
• Added test for checking grants in default schema (#470).
• Added unit tests for generic permissions (#457).
• Enabled integration tests via OIDC for every pull request (#378).
• Added check if permissions are up to date (#421).
• Fixed casing in all_tables.sql query. (#464).
• Fixed missed scans for empty databases and views in crawl_grants (#469).
• Improved logging colors for dark terminal backgrounds (#468).
• Improved local group migration state handling and made log files flush every 10 minutes (#449).
• Moved workspace listing as a separate task for an assessment workflow (#437).
• Removed rate limit for get or create backup group to speed up the prepare environment (#453).
• Updated databricks-sdk requirement from ~=0.10.0 to ~=0.11.0 (#448).

v0.4.0

• Added exception handling for secret scope not found. (#418).
• Added a crawler for creating an inventory of Azure Service Principals (#326).
• Added check if account group already exists during failure recovery (#446).
• Added checking for index out of range. (#429).
• Added hyperlink to UCX releases in the main readme (#408).
• Added integration test to check backup groups get deleted (#387).
• Added logging of errors during threadpool operations. (#376).
• Added recovery mode for workspace-local groups from temporary groups (#435).
• Added support for migrating Legacy Table ACLs from workspace-local to account-level groups (#412).
• Added detection for installations of unreleased versions (#399).
• Decoupled PermissionsManager from GroupMigrationToolkit (#407).
• Enabled debug logging for every job task run through a file, which is accessible from both workspace UI and Databricks CLI (#426).
• Ensured that table exists, even when crawlers produce zero records (#373).
• Extended test suite for HMS->HMS TACL migration (#439).
• Fixed handling of secret scope responses (#431).
• Fixed crawl_permissions task to respect 'workspace_start_path' config (#444).
• Fixed broken logic in parallel module and applied hardened error handling design for parallel code (#405).
• Fixed codecov.io reporting (#403).
• Fixed integration tests for crawlers (#379).
• Improved README.py and logging messages (#433).
• Improved cleanup for workspace backup groups by adding more retries on errors (#375).
• Improved dashboard queries to show unsupported storage types. (#398).
• Improved documentation for readme notebook (#257).
• Improved test coverage for installer (#371).
• Introduced deterministic env_or_skip fixture for integration tests (#396).
• Made HMS & UC fixtures return CatalogInfo, SchemaInfo, and TableInfo (#409).
• Merge workspace_access.Crawler and workspace_access.Applier interfaces to workspace_access.AclSupport (#436).
• Moved examples to docs (#404).
• Properly isolated integration testing for workflows on an existing shared cluster (#414).
• Removed thread pool for any IAM Group removals and additions (#394).
• Replace plus char with minus in version tag for GCP dev installation of UCX (#420).
• Run integration tests on shared clusters for a faster devloop (#397).
• Show difference between serverless and PRO warehouses during installation (#385).
• Split migrate-groups workflow into three different stages for reliability (#442).
• Use groups instead of usernames in code owners file (#389).