Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add primary key to preview_cards_statuses join table (includes deduplication migration) #25243

Conversation

ClearlyClaire
Copy link
Contributor

@ClearlyClaire ClearlyClaire commented Jun 2, 2023

Alternative to #25235.

Upsides:

  • much quicker migrations with no significant locking

Downsides:

  • new indexes are added in post-deployment migrations: this is fine, but this is a weird pattern that may cause issues for admins skipping releases if we build upon these
  • additional application-level locking code is added and required to ensure the post-deployment's migration termination, but only useful while the migrations hasn't run
  • while it's extremely unlikely, post-deployment migrations are not guaranteed to terminate if they are run while old code is running
  • still needlessly using a many-to-many relationship

@ClearlyClaire
Copy link
Contributor Author

ClearlyClaire commented Jun 2, 2023

Note that we could add application-level logic to prevent the race conditions, and move the deduplication and index creation to a post-deployment migration. This should solve the most serious downside of this approach, provided that admins do run the post-deployment migrations correctly.

@ClearlyClaire ClearlyClaire changed the title Add unique index on preview cards statuses Add primary key to preview_cards_statuses join table (includes deduplication migration) Jun 2, 2023
@ClearlyClaire ClearlyClaire force-pushed the migration/unique-index-on-preview_cards_statuses branch from 1a61102 to 62d401f Compare June 2, 2023 11:32
@github-actions
Copy link
Contributor

github-actions bot commented Jun 5, 2023

This pull request has merge conflicts that must be resolved before it can be merged.

def deduplicate_and_reindex!
deduplicate_preview_cards!

safety_assured { execute 'REINDEX INDEX preview_cards_statuses_pkey' }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will REINDEX actually work here? If the index on line 7 fails because of non-uniqueness, does the index still exist, or is it never created? I would assume the latter, but I'm not sure.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I somehow missed that this PR had a review. Indexes created with concurrently will exist but be marked as invalid, reindexing should work.

disable_ddl_transaction!

def up
ActiveRecord::Base.transaction do
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is a single query not equivalent to a transaction anyway?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should be equivalent. Not sure why I wrapped that in a transaction here.

@ClearlyClaire ClearlyClaire force-pushed the migration/unique-index-on-preview_cards_statuses branch from 62d401f to 51ce4d5 Compare July 17, 2023 16:11
@github-actions
Copy link
Contributor

This pull request has resolved merge conflicts and is ready for review.

@ClearlyClaire ClearlyClaire force-pushed the migration/unique-index-on-preview_cards_statuses branch from 51ce4d5 to 1432b5c Compare July 17, 2023 16:30
@renchap renchap added this to the 4.2.0 milestone Jul 21, 2023
@github-actions
Copy link
Contributor

This pull request has merge conflicts that must be resolved before it can be merged.

@ClearlyClaire ClearlyClaire force-pushed the migration/unique-index-on-preview_cards_statuses branch from 1432b5c to 7a55375 Compare August 3, 2023 07:42
@ClearlyClaire ClearlyClaire marked this pull request as ready for review August 3, 2023 07:42
@github-actions
Copy link
Contributor

github-actions bot commented Aug 3, 2023

This pull request has resolved merge conflicts and is ready for review.

@ClearlyClaire ClearlyClaire merged commit 6b896b2 into mastodon:main Aug 3, 2023
jsgoldstein added a commit to jsgoldstein/mastodon that referenced this pull request Aug 28, 2023
* Update dependency haml_lint to v0.49.2 (mastodon#26222)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Fix: Streaming server memory leak in HTTP EventSource cleanup (mastodon#26228)

* Swap debug statements in streaming server (mastodon#26231)

* Fix missing return values in streaming (mastodon#26233)

* Use original URL in preview if it redirects to 4xx page (mastodon#26200)

* Allow filtering for Chinese languages (mastodon#26066)

* Update README.md (mastodon#25435)

* Add end-to-end (system) tests (mastodon#25461)

* Rubocop fix: `Perfomance/UnfreezeString` (mastodon#26217)

* Update rubocop rules for linelength (mastodon#26190)

* Add coverage for `LanguageValidator` (mastodon#25593)

* Add coverage for `URLValidator` (mastodon#25591)

* Fix `RSpec/EmptyExampleGroup` cop (mastodon#24735)

* Profiling tools configuration improvement (mastodon#25383)

* Load `rspec-rails` gem in `test` + `development` (mastodon#25768)

* Ignore the brakeman `PermitAttributes` check (mastodon#25915)

* Fix public TL not indicating new toots when `onlyRemote` is enabled (mastodon#26247)

Signed-off-by: Plastikmensch <plastikmensch@users.noreply.github.com>

* Remove obsolete backport from Rails 7 (mastodon#26254)

* Change aspect ratios on link previews in web UI (mastodon#26250)

* Fix `Importer::BaseImporter#clean_up!` not using proper primary key (mastodon#26269)

* Bump version to v4.1.6 (mastodon#26272)

* Do not normalize URL before fetching it (mastodon#26219)

* Change /api/v1/peers/search to be case-insensitive when using Elasticsearch (mastodon#26268)

* Change interaction modal input to disable browser spell-checking, capitalization and autocomplete (mastodon#26267)

* Fix AVIF attachments (mastodon#26264)

* Add List-Unsubscribe email header (mastodon#26085)

* Fix wrong border radius on link cards in web UI (mastodon#26287)

* Fix line clamp for link previews in web UI (mastodon#26286)

* Change design of role badges in web UI (mastodon#26281)

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Fix request URL normalisation for bare domain and 8-bit characters (mastodon#26285)

* Refactor: replace whitelist_mode mentions with limited_federation_mode (mastodon#26252)

* change column link to add a better keyboard focus indicator (mastodon#26278)

* Fix crash in `tootctl status remove` and some old migrations (mastodon#26210)

* Add client-side timeout on resend confirmation button (mastodon#26300)

* Allow spaces around commas in ALLOWED_PRIVATE_ADDRESSES (mastodon#26297)

* Change indexing frequency from 5 minutes to 1 minute, add locks to schedulers (mastodon#26304)

* Add primary key to preview_cards_statuses join table (includes deduplication migration) (mastodon#25243)

* Add `GET /api/v1/instance/languages` to REST API (mastodon#24443)

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Update dependency aws-sdk-s3 to v1.132.0 (mastodon#26227)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Add alt text for preview card thumbnails (mastodon#26184)

* Update dependency rack to v2.2.8 (mastodon#26312)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency react-redux to v8.1.2 (mastodon#26314)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency test-prof to v1.2.2 (mastodon#26316)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency core-js to v3.32.0 (mastodon#26317)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency lograge to v0.13.0 (mastodon#26318)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Rename “read” database to “replica” for consistency (mastodon#26326)

* Update dependency selenium-webdriver to v4.10.0 (mastodon#26322)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency rack-attack to v6.7.0 (mastodon#26319)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Add direct link to the Single-Sign On provider if there is only one sign up method available (mastodon#26083)

* Re-add StatsD support through the `nsa` gem (mastodon#26310)

* Change reblogs to be excluded from "Posts and replies" tab in web UI (mastodon#26302)

* Change design of hidden media overlay (again) in web UI (mastodon#26330)

* Fix incorrect model annotation for List#exclusive (mastodon#26313)

* Fix light theme select option for hashtags (mastodon#26311)

* Fix confirmation when closing media edition modal with unsaved changes (mastodon#26342)

* Change streaming `/metrics` to include additional metrics (mastodon#26299)

* Fix missing cached preview cards attributes (mastodon#26343)

* Fix report modal secondary buttons style (mastodon#26341)

* Change header of hashtag timelines in web UI (mastodon#26362)

* New Crowdin Translations (automated) (mastodon#26209)

Co-authored-by: GitHub Actions <noreply@github.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Make mastodon-streaming systemd unit templated (mastodon#24751)

Co-authored-by: Emelia Smith <ThisIsMissEm@users.noreply.github.com>

* Spec media attachment speedups (mastodon#25416)

* Fix interaction modal layout (mastodon#26368)

* Fix list import concurrently creating lists of the same name (mastodon#26372)

* Omniauth 2.0 version bump (mastodon#24209)

* Change account search tokenizer and queries (mastodon#26378)

* Fix adding column with default value taking longer on Postgres >= 11 (mastodon#26375)

* Fix `preview_cards_statuses_pkey` not being reindexed concurrently (mastodon#26384)

* Change `DB_REPLICA_*` environment variables to `REPLICA_DB_*` (mastodon#26386)

* Fix Content Security Policy sometimes unnecessarily allowing hCaptcha scripts (mastodon#26388)

* Prepare v4.2.0-beta1 (mastodon#26339)

* Fix blocking subdomains of an already-blocked domain (mastodon#26392)

* Use migration classes in migrations where current definition conflicts with older (mastodon#26390)

* Restore console behavior in `test` env (mastodon#26401)

* Change: Block GPTBot (mastodon#26396)

* Avoid connecting to a running ES instance in ES search check spec (mastodon#26413)

* Change the hashtag column to not display the hashtag header on pinned columns (mastodon#26416)

* Update dependency sass to v1.64.2 (mastodon#26315)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update eslint (non-major) (mastodon#26323)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* New Crowdin Translations (automated) (mastodon#26373)

Co-authored-by: GitHub Actions <noreply@github.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Update dependency aws-sdk-s3 to v1.132.1 (mastodon#26423)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency immutable to v4.3.2 (mastodon#26425)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency haml_lint to v0.49.3 (mastodon#26424)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency eslint-config-prettier to v9 (mastodon#26434)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency sass to v1.65.1 (mastodon#26433)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency mime-types to '~> 3.5.0' (mastodon#26431)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency regenerator-runtime to ^0.14.0 (mastodon#26432)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency rails to v7.0.7 (mastodon#26428)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency @rails/ujs to v7.0.7 (mastodon#26422)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency pg to v8.11.2 (mastodon#26426)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update babel monorepo to v7.22.10 (mastodon#26421)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency eslint-import-resolver-typescript to v3.6.0 (mastodon#26429)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency pg-connection-string to v2.6.2 (mastodon#26427)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Add `S3_DISABLE_CHECKSUM_MODE` environment variable for compatibility with some S3-compatible providers (mastodon#26435)

* Merge duplicate Gemfile groups (mastodon#26441)

* Upgrade JS dev dependencies (mastodon#26442)

* Fix reply not preserving the language from the replied-to post (mastodon#26452)

* Add missing instances option to tootctl search deploy (mastodon#26461)

* Remove old non-unique index on preview_cards statuses join table (mastodon#26447)

* Upgrade `@types/react` (mastodon#26457)

* Fix "Create Account" button in interaction modal (mastodon#26459)

* Improve Renovate configuration (mastodon#26306)

* Update dependency faker to v3.2.1 (mastodon#26474)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency autoprefixer to v10.4.15 (mastodon#26473)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency nokogiri to v1.15.4 (mastodon#26476)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Fix `lint:json` not processing `json5` extension (mastodon#26481)

* New Crowdin Translations (automated) (mastodon#26444)

Co-authored-by: GitHub Actions <noreply@github.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Add `ES_PRESET` option to customize numbers of shards and replicas (mastodon#26483)

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>

* Fix `repo:changelog` task matching strings that are not Pull Request identifiers (mastodon#26280)

* Add privacy tab in profile settings (mastodon#26484)

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>

* Add support for `indexable` attribute on remote actors (mastodon#26485)

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>

* Fix ES_PRESET not being applied to Chewy's internal index (mastodon#26489)

* Fix unfollow icon styling in advanced column (mastodon#26482)

* Add display of out-of-band hashtags in the web interface (mastodon#26492)

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>

* Fix hashtag bar display when status is in a thread (mastodon#26497)

* Update dependency postcss to v8.4.28 (mastodon#26502)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Fix hashtag bar sometimes including tags that appear in the post's body (mastodon#26506)

* Fix “legal” report category not showing up in moderation interface (mastodon#26509)

* Change “privacy and reach” settings so that unchecking boxes always increase privacy and checking them always increase reach (mastodon#26508)

* Fix case-insensitive comparison of hashtags to do case-folding (mastodon#26525)

* Fix cached posts including stale stats (mastodon#26409)

* Update dependency pg to v8.11.3 (mastodon#26519)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Allow setting a custom HTTP method in CacheBuster (mastodon#26528)

Co-authored-by: Jorijn Schrijvershof <jorijn@jorijn.com>

* Fix `frame_rate` for videos where `ffprobe` reports 0/0 (mastodon#26500)

* Update dependency puma to v6.3.1 (mastodon#26537)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency @material-design-icons/svg to v0.14.11 (mastodon#26536)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Remove redundant ready() wrapper (mastodon#26533)

* Fix: support both DATABASE_URL and DB_PASS (mastodon#26295)

* Fix already initialized constant warning (mastodon#26542)

* Change follow recommendation materialized view to be faster in most cases (mastodon#26545)

Co-authored-by: Renaud Chaput <renchap@gmail.com>

* Fix profile picture preview (mastodon#26538)

* Add ability to delete avatar or header picture via the API (mastodon#25124)

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Do not start LibreTranslate and Elasticsearch on GitHub Codespaces (mastodon#26382)

* Update dependency core-js to v3.32.1 (mastodon#26548)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update eslint (non-major) (mastodon#26567)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Refactor `Api::V1::ProfilesController` into two separate controllers (mastodon#26573)

* Add auto-refresh of accounts we get new messages/edits of (mastodon#26510)

* Add Elasticsearch cluster health check and indexes mismatch check to dashboard (mastodon#26448)

* Remove hashtags from the last line of a status if it only contains hashtags (mastodon#26499)

* Bump version to v4.2.0-beta2 (mastodon#26579)

* Fix layout of the closed registrations modal (mastodon#26593)

* Update rubocop and rubocop-rspec (mastodon#26329)

* Add `circular-dependency-plugin` to detect any circular deps issues (mastodon#26600)

* Update `SECURITY.md` to indicate issues can be reported on Github (mastodon#26599)

* Update dependency mime-types to v3.5.1 (mastodon#26595)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Add cherokee to languages dropdown (mastodon#26012)

* Add Kalmyk to languages dropdown (mastodon#26013)

* Fix admin dashboard check when using Elasticsearch with `ES_PREFIX` (mastodon#26605)

* Better hashtag normalization when processing a post (mastodon#26614)

* Update dependency rails to v7.0.7.2 (mastodon#26612)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency react-textarea-autosize to v8.5.3 (mastodon#26607)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Add support for federating `memorial` attribute  (mastodon#26583)

* Fix unexpected audio stream transcoding when uploaded video is eligible to passthrough (mastodon#26608)

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Update Account Search to prioritize username over display name (mastodon#26623)

* Change the hashtag bar to be hidden when there is a CW and the post is not expanded (mastodon#26615)

* Fix some React warnings (mastodon#26609)

* Change hashtag bar tags to be de-emphasized (mastodon#26606)

* Update dependency immutable to v4.3.3 (mastodon#26622)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency oj to v3.16.0 (mastodon#26520)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update dependency sass to v1.66.1 (mastodon#26534)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Add elastic search installation into Vagrantfile (mastodon#26512)

* Fix timeout on invalid set of exclusionary parameters in `/api/v1/timelines/public` (mastodon#26239)

* Change opacity of the delete icon in the search field to be more visible (mastodon#26449)

* Update dependency rspec-sidekiq to v4 (mastodon#26627)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Consolidate inclusion of `admin` js pack link (mastodon#26628)

* Update dependency redis to v4.6.8 (mastodon#26630)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Fix: Workaround to load MASTODON_VERSION_* in Docker. (mastodon#26591)

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Fix selecting domains to forward reports to not passing the information correctly (mastodon#26636)

* Fix clicking “Explore” or “Live feeds” column headers to scroll in advanced mode (mastodon#26633)

Co-authored-by: Plastikmensch <Plastikmensch@users.noreply.github.com>

* Change admin e-mail notification settins to be their own settings group (mastodon#26596)

* Change nightlies versioning from `v4.2.0+2023-08-23` to `v4.2.0-nightly.2023-08-23` (mastodon#26626)

* Add new public status index (mastodon#26344)

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Fix changelog wording and missing items (mastodon#26638)

* Update babel monorepo to v7.22.11 (mastodon#26640)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Add `data-nosnippet` attribute to remote posts and local posts with `noindex` (mastodon#26648)

* Add PublicStatusesCheck to Elasticsearch index check on admin dashboard (mastodon#26650)

* Add Elasticsearch/OpenSearch version to “Software” in admin dashboard (mastodon#26652)

* Fix statuses search Elasticsearch query (mastodon#26657)

* Update dependency immutable to v4.3.4 (mastodon#26655)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update ordering to use `id` from body of document instead of deprecated `_id` (mastodon#26659)

* Fix dashboard check for Elasticsearch suggested command including incorrect names (mastodon#26658)

* Change the pre-release versioning scheme and associated environment variables (mastodon#26653)

* Fix toast saying "published" instead of "saved" after editing post in web UI (mastodon#26664)

* Fix nightly build version (mastodon#26676)

* Change queue of job when opting into search from `default` to `pull` (mastodon#26688)

* Fix unnecessary condition causing seqscan when indexing (mastodon#26689)

* Change indexing jobs to use database replica (mastodon#26692)

* Update eslint (non-major) (mastodon#26694)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Fix not being able to negate prefix clauses in search (mastodon#26672)

* Update dependency haml_lint to v0.50.0 (mastodon#26665)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Update DefinitelyTyped types (non-major) (mastodon#26693)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

* Fix multiple issues with status index mappings (mastodon#26686)

* Update dependency @testing-library/jest-dom to v6 (mastodon#26479)

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Renaud Chaput <renchap@gmail.com>

* Add `from:me` syntax to search (mastodon#26660)

* Add search options to search popout in web UI (mastodon#26662)

* Fix incorrect call to `PublicStatusesIndex.import` (mastodon#26697)

* A first pass at adding some basic ES tests

* Add more tests

---------

Signed-off-by: Plastikmensch <plastikmensch@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Emelia Smith <ThisIsMissEm@users.noreply.github.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Co-authored-by: Renaud Chaput <renchap@gmail.com>
Co-authored-by: Christian Schmidt <github@chsc.dk>
Co-authored-by: gunchleoc <fios@foramnagaidhlig.net>
Co-authored-by: Jay Prakash Kalia <jaykalia047@gmail.com>
Co-authored-by: Matt Jankowski <matt@jankowski.online>
Co-authored-by: Plastikmensch <Plastikmensch@users.noreply.github.com>
Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Trevor Wolf <teeerevor@gmail.com>
Co-authored-by: Val Lorentz <progval+github@progval.net>
Co-authored-by: Daniel M Brasil <danielmbrasil@protonmail.com>
Co-authored-by: CSDUMMI <31551856+CSDUMMI@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Actions <noreply@github.com>
Co-authored-by: Eashwar Ranganathan <eranganathan@lyft.com>
Co-authored-by: Foritus <rich@aornis.com>
Co-authored-by: Nick Schonning <nschonni@gmail.com>
Co-authored-by: Jeong Arm <kjwonmail@gmail.com>
Co-authored-by: mogaminsk <mgmnjp@icloud.com>
Co-authored-by: HonkingGoose <34918129+HonkingGoose@users.noreply.github.com>
Co-authored-by: Santiago Kozak <santikzk1406@gmail.com>
Co-authored-by: Jorijn Schrijvershof <jorijn@jorijn.com>
Co-authored-by: Nicolai Søborg <NicolaiSoeborg@users.noreply.github.com>
Co-authored-by: Robert R George <rgeorge@midnightweb.net>
Co-authored-by: yufushiro <62991447+yufushiro@users.noreply.github.com>
Co-authored-by: Antonin Del Fabbro <30950182+AntoninDelFabbro@users.noreply.github.com>
Co-authored-by: Jaehong Kang <sinoru@me.com>
nrdufour added a commit to nrdufour/home-ops that referenced this pull request Oct 3, 2023
…103)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [ghcr.io/mastodon/mastodon](https://github.com/mastodon/mastodon) | minor | `v4.1.9` -> `v4.2.0` |

---

### Release Notes

<details>
<summary>mastodon/mastodon (ghcr.io/mastodon/mastodon)</summary>

### [`v4.2.0`](https://github.com/mastodon/mastodon/releases/tag/v4.2.0)

[Compare Source](https://github.com/mastodon/mastodon/compare/v4.1.9...v4.2.0)

<h1><picture>
  <source media="(prefers-color-scheme: dark)" srcset="./lib/assets/wordmark.dark.png?raw=true">
  <source media="(prefers-color-scheme: light)" srcset="./lib/assets/wordmark.light.png?raw=true">
  <img alt="Mastodon" src="./lib/assets/wordmark.light.png?raw=true" height="34">
</picture></h1>

We are excited to release Mastodon 4.2.0, our next major upgrade 🎉

In this version, we've added (opt-in) search for posts, along with a refined set of privacy options where users can decide whether to have their content indexed for discovery. There are also many other small improvements across the user interface that come together to make everything much more polished and smooth to get started.

To get an overview of what this release brings, check out [our announcement blog post](https://blog.joinmastodon.org/2023/09/mastodon-4.2/).

> **:warning: We recently released important security updates.**
>
> If you don't want or can't update to 4.2.0 yet, the security updates are also available for the [4.1.x branch](https://github.com/mastodon/mastodon/releases/tag/v4.1.9), the [4.0.x branch](https://github.com/mastodon/mastodon/releases/tag/v4.0.11) and the [3.5.x branch](https://github.com/mastodon/mastodon/releases/tag/v3.5.14).

##### Upgrade overview

This release contains upgrade notes that deviate from the norm:

ℹ️ Requires streaming API restart
ℹ️ Requires database migrations
ℹ️ Starting from this release, Mastodon will periodically check for updates (see below if you want to disable that behavior)
:warning: Requires rebuilding Elasticsearch indexes
:warning: The minimal supported version for Ruby has been bumped to 3.0
:warning: The minimal supported version for Node.js has been bumped to 16
:warning: The minimal supported version for PostgreSQL has been bumped to 10. Please note that using PostgreSQL 10 or 11 is deprecated and will not be supported in 4.3.0.
:warning: The minimal supported version for LibreTranslate has been bumped to 1.3.3
:warning: The way database replicas are configured has changed
:warning: Disables part of the StatsD integration by default
:warning: Drops built-in clustering support from the streaming server
:warning: Updated systemd unit files for the streaming server
:warning: We will stop bundling PgHero in a future release

For more information, scroll down to the upgrade instructions section.

##### Changelog

The following changelog entries focus on changes visible to users, administrators, client developers or federated software developers, but there has also been a lot of code modernization, refactoring, and tooling work, in particular by [@&#8203;danielmbrasil](https://github.com/danielmbrasil), [@&#8203;mjankowski](https://github.com/mjankowski), [@&#8203;nschonni](https://github.com/nschonni), [@&#8203;renchap](https://github.com/renchap), and [@&#8203;takayamaki](https://github.com/takayamaki).

##### Added

-   **Add full-text search of opted-in public posts and rework search operators** ([Gargron](https://github.com/mastodon/mastodon/pull/26485), [jsgoldstein](https://github.com/mastodon/mastodon/pull/26344), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26657), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26650), [jsgoldstein](https://github.com/mastodon/mastodon/pull/26659), [Gargron](https://github.com/mastodon/mastodon/pull/26660), [Gargron](https://github.com/mastodon/mastodon/pull/26663), [Gargron](https://github.com/mastodon/mastodon/pull/26688), [Gargron](https://github.com/mastodon/mastodon/pull/26689), [Gargron](https://github.com/mastodon/mastodon/pull/26686), [Gargron](https://github.com/mastodon/mastodon/pull/26687), [Gargron](https://github.com/mastodon/mastodon/pull/26692), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26697), [Gargron](https://github.com/mastodon/mastodon/pull/26699), [Gargron](https://github.com/mastodon/mastodon/pull/26701), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26710), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26739), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26754), [Gargron](https://github.com/mastodon/mastodon/pull/26662), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26755), [Gargron](https://github.com/mastodon/mastodon/pull/26781), [Gargron](https://github.com/mastodon/mastodon/pull/26782), [Gargron](https://github.com/mastodon/mastodon/pull/26760), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26756), [Gargron](https://github.com/mastodon/mastodon/pull/26784), [Gargron](https://github.com/mastodon/mastodon/pull/26807), [Gargron](https://github.com/mastodon/mastodon/pull/26835), [Gargron](https://github.com/mastodon/mastodon/pull/26847), [Gargron](https://github.com/mastodon/mastodon/pull/26834), [arbolitoloco1](https://github.com/mastodon/mastodon/pull/26893), [tribela](https://github.com/mastodon/mastodon/pull/26896), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26927), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26959), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/27014))
    This introduces a new `public_statuses` Elasticsearch index for public posts by users who have opted in to their posts being searchable (`toot#indexable` flag).
    This also revisits the other indexes to provide more useful indexing, and adds new search operators such as `from:me`, `before:2022-11-01`, `after:2022-11-01`, `during:2022-11-01`, `language:fr`, `has:poll`, or `in:library` (for searching only in posts you have written or interacted with).
    Results are now ordered chronologically.
-   **Add admin notifications for new Mastodon versions** ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26582))
    This is done by querying `https://api.joinmastodon.org/update-check` every 30 minutes in a background job.
    That URL can be changed using the `UPDATE_CHECK_URL` environment variable, and the feature outright disabled by setting that variable to an empty string (`UPDATE_CHECK_URL=`).
-   **Add “Privacy and reach” tab in profile settings** ([Gargron](https://github.com/mastodon/mastodon/pull/26484), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26508))
    This reorganized scattered privacy and reach settings to a single place, as well as improve their wording.
-   **Add display of out-of-band hashtags in the web interface** ([Gargron](https://github.com/mastodon/mastodon/pull/26492), [arbolitoloco1](https://github.com/mastodon/mastodon/pull/26497), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26506), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26525), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26606), [Gargron](https://github.com/mastodon/mastodon/pull/26666), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26960))
-   **Add role badges to the web interface** ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25649), [Gargron](https://github.com/mastodon/mastodon/pull/26281))
-   **Add ability to pick domains to forward reports to using the `forward_to_domains` parameter in `POST /api/v1/reports`** ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25866), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26636))
    The `forward_to_domains` REST API parameter is a list of strings. If it is empty or omitted, the previous behavior is maintained.
    The `forward` parameter still needs to be set for `forward_to_domains` to be taken into account.
    The forwarded-to domains can only include that of the original author and people being replied to.
-   **Add forwarding of reported replies to servers being replied to** ([Gargron](https://github.com/mastodon/mastodon/pull/25341), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26189))
-   Add `ONE_CLICK_SSO_LOGIN` environment variable to directly link to the Single-Sign On provider if there is only one sign up method available ([CSDUMMI](https://github.com/mastodon/mastodon/pull/26083), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26368), [CSDUMMI](https://github.com/mastodon/mastodon/pull/26857), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26901))
-   **Add webhook templating** ([Gargron](https://github.com/mastodon/mastodon/pull/23289))
-   **Add webhooks for local `status.created`, `status.updated`, `account.updated` and `report.updated`** ([VyrCossont](https://github.com/mastodon/mastodon/pull/24133), [VyrCossont](https://github.com/mastodon/mastodon/pull/24243), [VyrCossont](https://github.com/mastodon/mastodon/pull/24211))
-   **Add exclusive lists** ([dariusk, necropolina](https://github.com/mastodon/mastodon/pull/22048), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25324))
-   **Add a confirmation screen when suspending a domain** ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25144), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25603))
-   **Add support for importing lists** ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25203), [mgmn](https://github.com/mastodon/mastodon/pull/26120), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26372))
-   **Add optional hCaptcha support** ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25019), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25057), [Gargron](https://github.com/mastodon/mastodon/pull/25395), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26388))
-   **Add lines to threads in web UI** ([Gargron](https://github.com/mastodon/mastodon/pull/24549), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24677), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24696), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24711), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24714), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24713), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24715), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24800), [teeerevor](https://github.com/mastodon/mastodon/pull/25706), [renchap](https://github.com/mastodon/mastodon/pull/25807))
-   **Add new onboarding flow to web UI** ([Gargron](https://github.com/mastodon/mastodon/pull/24619), [Gargron](https://github.com/mastodon/mastodon/pull/24646), [Gargron](https://github.com/mastodon/mastodon/pull/24705), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24872), [ThisIsMissEm](https://github.com/mastodon/mastodon/pull/24883), [Gargron](https://github.com/mastodon/mastodon/pull/24954), [stevenjlm](https://github.com/mastodon/mastodon/pull/24959), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25010), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25275), [Gargron](https://github.com/mastodon/mastodon/pull/25559), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25561))
-   **Add auto-refresh of accounts we get new messages/edits of** ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26510))
-   **Add Elasticsearch cluster health check and indexes mismatch check to dashboard** ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26448), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26605), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26658))
-   Add `hide_collections`, `discoverable` and `indexable` attributes to credentials API ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26998))
-   Add `S3_ENABLE_CHECKSUM_MODE` environment variable to enable checksum verification on compatible S3-providers ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26435))
-   Add admin API for managing tags ([rrgeorge](https://github.com/mastodon/mastodon/pull/26872))
-   Add a link to hashtag timelines from the Trending hashtags moderation interface ([gunchleoc](https://github.com/mastodon/mastodon/pull/26724))
-   Add timezone to datetimes in e-mails ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26822))
-   Add `authorized_fetch` server setting in addition to env var ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25798), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26958))
-   Add avatar image to webfinger responses ([tvler](https://github.com/mastodon/mastodon/pull/26558))
-   Add debug logging on signature verification failure ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26637), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26812))
-   Add explicit error messages when DeepL quota is exceeded ([lutoma](https://github.com/mastodon/mastodon/pull/26704))
-   Add Elasticsearch/OpenSearch version to “Software” in admin dashboard ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26652))
-   Add `data-nosnippet` attribute to remote posts and local posts with `noindex` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26648))
-   Add support for federating `memorial` attribute ([rrgeorge](https://github.com/mastodon/mastodon/pull/26583))
-   Add Cherokee and Kalmyk to languages dropdown ([gunchleoc](https://github.com/mastodon/mastodon/pull/26012), [gunchleoc](https://github.com/mastodon/mastodon/pull/26013))
-   Add `DELETE /api/v1/profile/avatar` and `DELETE /api/v1/profile/header` to the REST API ([danielmbrasil](https://github.com/mastodon/mastodon/pull/25124), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26573))
-   Add `ES_PRESET` option to customize numbers of shards and replicas ([Gargron](https://github.com/mastodon/mastodon/pull/26483), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26489))
    This can have a value of `single_node_cluster` (default), `small_cluster` (uses one replica) or `large_cluster` (uses one replica and a higher number of shards).
-   Add `CACHE_BUSTER_HTTP_METHOD` environment variable ([renchap](https://github.com/mastodon/mastodon/pull/26528), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26542))
-   Add support for `DB_PASS` when using `DATABASE_URL` ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/26295))
-   Add `GET /api/v1/instance/languages` to REST API ([danielmbrasil](https://github.com/mastodon/mastodon/pull/24443))
-   Add primary key to `preview_cards_statuses` join table ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25243), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26384), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26447), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26737), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26979))
-   Add client-side timeout on resend confirmation button ([Gargron](https://github.com/mastodon/mastodon/pull/26300))
-   Add published date and author to news on the explore screen in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/26155))
-   Add `lang` attribute to various UI components ([c960657](https://github.com/mastodon/mastodon/pull/23869), [c960657](https://github.com/mastodon/mastodon/pull/23891), [c960657](https://github.com/mastodon/mastodon/pull/26111), [c960657](https://github.com/mastodon/mastodon/pull/26149))
-   Add stricter protocol fields validation for accounts ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25937))
-   Add support for Azure blob storage ([mistydemeo](https://github.com/mastodon/mastodon/pull/23607), [mistydemeo](https://github.com/mastodon/mastodon/pull/26080))
-   Add toast with option to open post after publishing in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/25564), [Signez](https://github.com/mastodon/mastodon/pull/25919), [Gargron](https://github.com/mastodon/mastodon/pull/26664))
-   Add canonical link tags in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/25715))
-   Add button to see results for polls in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/25726))
-   Add at-symbol prepended to mention span title ([forsamori](https://github.com/mastodon/mastodon/pull/25684))
-   Add users index on `unconfirmed_email` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25672), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25702))
-   Add superapp index on `oauth_applications` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25670))
-   Add index to backups on `user_id` column ([mjankowski](https://github.com/mastodon/mastodon/pull/25647))
-   Add onboarding prompt when home feed too slow in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/25267), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25556), [Gargron](https://github.com/mastodon/mastodon/pull/25579), [renchap](https://github.com/mastodon/mastodon/pull/25580), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25581), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25617), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25917), [Gargron](https://github.com/mastodon/mastodon/pull/26829), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26935))
-   Add `POST /api/v1/conversations/:id/unread` API endpoint to mark a conversation as unread ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25509))
-   Add `translate="no"` to outgoing mentions and links ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25524))
-   Add unsubscribe link and headers to e-mails ([Gargron](https://github.com/mastodon/mastodon/pull/25378), [c960657](https://github.com/mastodon/mastodon/pull/26085))
-   Add logging of websocket send errors ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/25280))
-   Add time zone preference ([Gargron](https://github.com/mastodon/mastodon/pull/25342), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26025))
-   Add `legal` as report category ([Gargron](https://github.com/mastodon/mastodon/pull/23941), [renchap](https://github.com/mastodon/mastodon/pull/25400), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26509))
-   Add `data-nosnippet` so Google doesn't use trending posts in snippets for `/` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25279))
-   Add card with who invited you to join when displaying rules on sign-up ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23475))
-   Add missing primary keys to `accounts_tags` and `statuses_tags` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25210))
-   Add support for custom sign-up URLs ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25014), [renchap](https://github.com/mastodon/mastodon/pull/25108), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25190), [mgmn](https://github.com/mastodon/mastodon/pull/25531))
    This is set using `SSO_ACCOUNT_SIGN_UP` and reflected in the REST API by adding `registrations.sign_up_url` to the `/api/v2/instance` endpoint.
-   Add polling and automatic redirection to `/start` on email confirmation ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25013))
-   Add ability to block sign-ups from IP using the CLI ([danielmbrasil](https://github.com/mastodon/mastodon/pull/24870))
-   Add ALT badges to media that has alternative text in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/24782), [c960657](https://github.com/mastodon/mastodon/pull/26166)
-   Add ability to include accounts with pending follow requests in lists ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/19727), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24810))
-   Add trend management to admin API ([rrgeorge](https://github.com/mastodon/mastodon/pull/24257))
    -   `POST /api/v1/admin/trends/statuses/:id/approve`
    -   `POST /api/v1/admin/trends/statuses/:id/reject`
    -   `POST /api/v1/admin/trends/links/:id/approve`
    -   `POST /api/v1/admin/trends/links/:id/reject`
    -   `POST /api/v1/admin/trends/tags/:id/approve`
    -   `POST /api/v1/admin/trends/tags/:id/reject`
    -   `GET /api/v1/admin/trends/links/publishers`
    -   `POST /api/v1/admin/trends/links/publishers/:id/approve`
    -   `POST /api/v1/admin/trends/links/publishers/:id/reject`
-   Add user handle to notification mail recipient address ([HeitorMC](https://github.com/mastodon/mastodon/pull/24240))
-   Add progress indicator to sign-up flow ([Gargron](https://github.com/mastodon/mastodon/pull/24545))
-   Add client-side validation for taken username in sign-up form ([Gargron](https://github.com/mastodon/mastodon/pull/24546))
-   Add `--approve` option to `tootctl accounts create` ([danielmbrasil](https://github.com/mastodon/mastodon/pull/24533))
-   Add “In Memoriam” banner back to profiles ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23591), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/23614))
    This adds the `memorial` attribute to the `Account` REST API entity.
-   Add colour to follow button when hashtag is being followed ([c960657](https://github.com/mastodon/mastodon/pull/24361))
-   Add further explanations to the profile link verification instructions ([drzax](https://github.com/mastodon/mastodon/pull/19723))
-   Add a link to Identity provider's account settings from the account settings ([CSDUMMI](https://github.com/mastodon/mastodon/pull/24100), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24628))
-   Add support for streaming server to connect to postgres with self-signed certs through the `sslmode` URL parameter ([ramuuns](https://github.com/mastodon/mastodon/pull/21431))
-   Add support for specifying S3 storage classes through the `S3_STORAGE_CLASS` environment variable ([hyl](https://github.com/mastodon/mastodon/pull/22480))
-   Add support for incoming rich text ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23913))
-   Add support for Ruby 3.2 ([tenderlove](https://github.com/mastodon/mastodon/pull/22928), [casperisfine](https://github.com/mastodon/mastodon/pull/24142), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24202), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26934))
-   Add API parameter to safeguard unexpected mentions in new posts ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/18350))

##### Changed

-   **Change hashtags to be displayed separately when they are the last line of a post** ([renchap](https://github.com/mastodon/mastodon/pull/26499), [renchap](https://github.com/mastodon/mastodon/pull/26614), [renchap](https://github.com/mastodon/mastodon/pull/26615))
-   **Change reblogs to be excluded from "Posts and replies" tab in web UI** ([Gargron](https://github.com/mastodon/mastodon/pull/26302))
-   **Change interaction modal in web interface** ([Gargron, ClearlyClaire](https://github.com/mastodon/mastodon/pull/26075), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26269), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26268), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26267), [mgmn](https://github.com/mastodon/mastodon/pull/26459), [tribela](https://github.com/mastodon/mastodon/pull/26461), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26593), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26795))
-   **Change design of link previews in web UI** ([Gargron](https://github.com/mastodon/mastodon/pull/26136), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26151), [Gargron](https://github.com/mastodon/mastodon/pull/26153), [Gargron](https://github.com/mastodon/mastodon/pull/26250), [Gargron](https://github.com/mastodon/mastodon/pull/26287), [Gargron](https://github.com/mastodon/mastodon/pull/26286), [c960657](https://github.com/mastodon/mastodon/pull/26184))
-   **Change "direct message" nomenclature to "private mention" in web UI** ([Gargron](https://github.com/mastodon/mastodon/pull/24248))
-   **Change translation feature to cover Content Warnings, poll options and media descriptions** ([c960657](https://github.com/mastodon/mastodon/pull/24175), [S-H-GAMELINKS](https://github.com/mastodon/mastodon/pull/25251), [c960657](https://github.com/mastodon/mastodon/pull/26168), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26452))
-   **Change account search to match by text when opted-in** ([jsgoldstein](https://github.com/mastodon/mastodon/pull/25599), [Gargron](https://github.com/mastodon/mastodon/pull/26378))
-   **Change import feature to be clearer, less error-prone and more reliable** ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/21054), [mgmn](https://github.com/mastodon/mastodon/pull/24874))
-   **Change local and federated timelines to be tabs of a single “Live feeds” column** ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25641), [Gargron](https://github.com/mastodon/mastodon/pull/25683), [mgmn](https://github.com/mastodon/mastodon/pull/25694), [Plastikmensch](https://github.com/mastodon/mastodon/pull/26247), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26633))
-   **Change user archive export to be faster and more reliable, and export `.zip` archives instead of `.tar.gz` ones** ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23360), [TheEssem](https://github.com/mastodon/mastodon/pull/25034))
-   **Change `mastodon-streaming` systemd unit files to be templated** ([e-nomem](https://github.com/mastodon/mastodon/pull/24751))
-   **Change `statsd` integration to disable sidekiq metrics by default** ([mjankowski](https://github.com/mastodon/mastodon/pull/25265), [mjankowski](https://github.com/mastodon/mastodon/pull/25336), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26310))
    This deprecates `statsd` support and disables the sidekiq integration unless `STATSD_SIDEKIQ` is set to `true`.
    This is because the `nsa` gem is unmaintained, and its sidekiq integration is known to add very significant overhead.
    Later versions of Mastodon will have other ways to get the same metrics.
-   **Change replica support to native Rails adapter** ([krainboltgreene](https://github.com/mastodon/mastodon/pull/25693), [Gargron](https://github.com/mastodon/mastodon/pull/25849), [Gargron](https://github.com/mastodon/mastodon/pull/25874), [Gargron](https://github.com/mastodon/mastodon/pull/25851), [Gargron](https://github.com/mastodon/mastodon/pull/25977), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26074), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26326), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26386), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26856))
    This is a breaking change, dropping `makara` support, and requiring you to update your database configuration if you are using replicas.
    To tell Mastodon to use a read replica, you can either set the `REPLICA_DB_NAME` environment variable (along with `REPLICA_DB_USER`, `REPLICA_DB_PASS`, `REPLICA_DB_HOST`, and `REPLICA_DB_PORT`, if they differ from the primary database), or the `REPLICA_DATABASE_URL` environment variable if your configuration is based on `DATABASE_URL`.
-   Change DCT method used for JPEG encoding to float ([electroCutie](https://github.com/mastodon/mastodon/pull/26675))
-   Change from `node-redis` to `ioredis` for streaming ([gmemstr](https://github.com/mastodon/mastodon/pull/26581))
-   Change private statuses index to index without crutches ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26713))
-   Change video compression parameters ([Gargron](https://github.com/mastodon/mastodon/pull/26631), [Gargron](https://github.com/mastodon/mastodon/pull/26745), [Gargron](https://github.com/mastodon/mastodon/pull/26766), [Gargron](https://github.com/mastodon/mastodon/pull/26970))
-   Change admin e-mail notification settings to be their own settings group ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26596))
-   Change opacity of the delete icon in the search field to be more visible ([AntoninDelFabbro](https://github.com/mastodon/mastodon/pull/26449))
-   Change Account Search to prioritize username over display name ([jsgoldstein](https://github.com/mastodon/mastodon/pull/26623))
-   Change follow recommendation materialized view to be faster in most cases ([renchap, ClearlyClaire](https://github.com/mastodon/mastodon/pull/26545))
-   Change `robots.txt` to block GPTBot ([Foritus](https://github.com/mastodon/mastodon/pull/26396))
-   Change header of hashtag timelines in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/26362), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26416))
-   Change streaming `/metrics` to include additional metrics ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/26299), [ThisIsMissEm](https://github.com/mastodon/mastodon/pull/26945))
-   Change indexing frequency from 5 minutes to 1 minute, add locks to schedulers ([Gargron](https://github.com/mastodon/mastodon/pull/26304))
-   Change column link to add a better keyboard focus indicator ([teeerevor](https://github.com/mastodon/mastodon/pull/26278))
-   Change poll form element colors to fit with the rest of the ui ([teeerevor](https://github.com/mastodon/mastodon/pull/26139), [teeerevor](https://github.com/mastodon/mastodon/pull/26162), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26164))
-   Change 'favourite' to 'favorite' for American English ([marekr](https://github.com/mastodon/mastodon/pull/24667), [gunchleoc](https://github.com/mastodon/mastodon/pull/26009), [nabijaczleweli](https://github.com/mastodon/mastodon/pull/26109))
-   Change ActivityStreams representation of suspended accounts to not use a blank `name` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25276))
-   Change focus UI for keyboard only input ([teeerevor](https://github.com/mastodon/mastodon/pull/25935), [Gargron](https://github.com/mastodon/mastodon/pull/26125), [Gargron](https://github.com/mastodon/mastodon/pull/26767))
-   Change thread view to scroll to the selected post rather than the post being replied to ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24685))
-   Change links in multi-column mode so tabs are open in single-column mode ([Signez](https://github.com/mastodon/mastodon/pull/25893), [Signez](https://github.com/mastodon/mastodon/pull/26070), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25973), [Signez](https://github.com/mastodon/mastodon/pull/26019), [Signez](https://github.com/mastodon/mastodon/pull/26759))
-   Change searching with `#` to include account index ([jsgoldstein](https://github.com/mastodon/mastodon/pull/25638))
-   Change label and design of sensitive and unavailable media in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/25712), [Gargron](https://github.com/mastodon/mastodon/pull/26135), [Gargron](https://github.com/mastodon/mastodon/pull/26330))
-   Change button colors to increase hover/focus contrast and consistency ([teeerevor](https://github.com/mastodon/mastodon/pull/25677), [Gargron](https://github.com/mastodon/mastodon/pull/25679))
-   Change dropdown icon above compose form from ellipsis to bars in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/25661))
-   Change header backgrounds to use fewer different colors in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/25577))
-   Change files to be deleted in batches instead of one-by-one ([Gargron](https://github.com/mastodon/mastodon/pull/23302), [S-H-GAMELINKS](https://github.com/mastodon/mastodon/pull/25586), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25587))
-   Change emoji picker icon ([iparr](https://github.com/mastodon/mastodon/pull/25479))
-   Change edit profile page ([Gargron](https://github.com/mastodon/mastodon/pull/25413), [c960657](https://github.com/mastodon/mastodon/pull/26538))
-   Change "bot" label to "automated" ([Gargron](https://github.com/mastodon/mastodon/pull/25356))
-   Change design of dropdowns in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/25107))
-   Change wording of “Content cache retention period” setting to highlight destructive implications ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23261))
-   Change autolinking to allow carets in URL search params ([renchap](https://github.com/mastodon/mastodon/pull/25216))
-   Change share action from being in action bar to being in dropdown in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/25105))
-   Change sessions to be ordered from most-recent to least-recently updated ([frankieroberto](https://github.com/mastodon/mastodon/pull/25005))
-   Change vacuum scheduler to also delete expired tokens and unused application records ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24868), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24871))
-   Change "Sign in" to "Login" ([Gargron](https://github.com/mastodon/mastodon/pull/24942))
-   Change domain suspensions to also be checked before trying to fetch unknown remote resources ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24535))
-   Change media components to use aspect-ratio rather than compute height themselves ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24686), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24943), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26801))
-   Change logo version in header based on screen size in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/24707))
-   Change label from "For you" to "People" on explore screen in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/24706))
-   Change logged-out WebUI HTML pages to be cached for a few seconds ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24708))
-   Change unauthenticated responses to be cached in REST API ([Gargron](https://github.com/mastodon/mastodon/pull/24348), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24662), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24665))
-   Change HTTP caching logic ([Gargron](https://github.com/mastodon/mastodon/pull/24347), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24604))
-   Change hashtags and mentions in bios to open in-app in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/24643))
-   Change styling of the recommended accounts to allow bio to be more visible ([chike00](https://github.com/mastodon/mastodon/pull/24480))
-   Change account search in moderation interface to allow searching by username including the leading `@` ([HeitorMC](https://github.com/mastodon/mastodon/pull/24242))
-   Change all components to use the same error page in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/24512))
-   Change search pop-out in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/24305))
-   Change user settings to be stored in a more optimal way ([Gargron](https://github.com/mastodon/mastodon/pull/23630), [c960657](https://github.com/mastodon/mastodon/pull/24321), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24453), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24460), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24558), [Gargron](https://github.com/mastodon/mastodon/pull/24761), [Gargron](https://github.com/mastodon/mastodon/pull/24783), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25508), [jsgoldstein](https://github.com/mastodon/mastodon/pull/25340), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26884), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/27012))
-   Change media upload limits and remove client-side resizing ([Gargron](https://github.com/mastodon/mastodon/pull/23726))
-   Change design of account rows in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/24247), [Gargron](https://github.com/mastodon/mastodon/pull/24343), [Gargron](https://github.com/mastodon/mastodon/pull/24956), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25131))
-   Change log-out to use Single Logout when using external log-in through OIDC ([CSDUMMI](https://github.com/mastodon/mastodon/pull/24020))
-   Change sidekiq-bulk's batch size from 10,000 to 1,000 jobs in one Redis call ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24034))
-   Change translation to only be offered for supported languages ([c960657](https://github.com/mastodon/mastodon/pull/23879), [c960657](https://github.com/mastodon/mastodon/pull/24037))
    This adds the `/api/v1/instance/translation_languages` REST API endpoint that returns an object with the supported translation language pairs in the form:
    ```json
    {
      "fr": ["en", "de"]
    }
    ```
    (where `fr` is a supported source language and `en` and `de` or supported output language when translating a `fr` string)
-   Change compose form checkbox to native input with `appearance: none` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/22949))
-   Change posts' clickable area to be larger ([c960657](https://github.com/mastodon/mastodon/pull/23621))
-   Change `followed_by` link to `location=all` if account is local on /admin/accounts/:id page ([tribela](https://github.com/mastodon/mastodon/pull/23467))

##### Removed

-   **Remove support for Node.js 14** ([renchap](https://github.com/mastodon/mastodon/pull/25198))
-   **Remove support for Ruby 2.7** ([nschonni](https://github.com/mastodon/mastodon/pull/24237))
-   **Remove clustering from streaming API** ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/24655))
-   **Remove anonymous access to the streaming API** ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23989))
-   Remove obfuscation of reply count in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/26768))
-   Remove `kmr` from language selection, as it was a duplicate for `ku` ([gunchleoc](https://github.com/mastodon/mastodon/pull/26014), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26787))
-   Remove 16:9 cropping from web UI ([Gargron](https://github.com/mastodon/mastodon/pull/26132))
-   Remove back button from bookmarks, favourites and lists screens in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/26126))
-   Remove display name input from sign-up form ([Gargron](https://github.com/mastodon/mastodon/pull/24704))
-   Remove `tai` locale ([c960657](https://github.com/mastodon/mastodon/pull/23880))
-   Remove empty Kushubian (csb) local files ([nschonni](https://github.com/mastodon/mastodon/pull/24151))
-   Remove `Permissions-Policy` header from all responses ([Gargron](https://github.com/mastodon/mastodon/pull/24124))

##### Fixed

-   **Fix filters not being applying in the explore page** ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25887))
-   **Fix being unable to load past a full page of filtered posts in Home timeline** ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24930))
-   **Fix log-in flow when involving both OAuth and external authentication** ([CSDUMMI](https://github.com/mastodon/mastodon/pull/24073))
-   **Fix broken links in account gallery** ([c960657](https://github.com/mastodon/mastodon/pull/24218))
-   **Fix migration handler not updating lists** ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24808))
-   Fix crash when viewing a moderation appeal and the moderator account has been deleted ([xrobau](https://github.com/mastodon/mastodon/pull/25900))
-   Fix error in Web UI when server rules cannot be fetched ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26957))
-   Fix paragraph margins resulting in irregular read-more cut-off in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/26828))
-   Fix notification permissions being requested immediately after login ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26472))
-   Fix performances of profile directory ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26840), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26842))
-   Fix mute button and volume slider feeling disconnected in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/26827), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26860))
-   Fix “Scoped order is ignored, it's forced to be batch order.” warnings ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26793))
-   Fix blocked domain appearing in account feeds ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26823))
-   Fix invalid `Content-Type` header for WebP images ([c960657](https://github.com/mastodon/mastodon/pull/26773))
-   Fix minor inefficiencies in `tootctl search deploy` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26721))
-   Fix filter form in profiles directory overflowing instead of wrapping ([arbolitoloco1](https://github.com/mastodon/mastodon/pull/26682))
-   Fix sign up steps progress layout in right-to-left locales ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26728))
-   Fix bug with “favorited by” and “reblogged by“ view on posts only showing up to 40 items ([timothyjrogers](https://github.com/mastodon/mastodon/pull/26577), [timothyjrogers](https://github.com/mastodon/mastodon/pull/26574))
-   Fix bad search type heuristic ([Gargron](https://github.com/mastodon/mastodon/pull/26673))
-   Fix not being able to negate prefix clauses in search ([Gargron](https://github.com/mastodon/mastodon/pull/26672))
-   Fix timeout on invalid set of exclusionary parameters in `/api/v1/timelines/public` ([danielmbrasil](https://github.com/mastodon/mastodon/pull/26239))
-   Fix adding column with default value taking longer on Postgres >= 11 ([Gargron](https://github.com/mastodon/mastodon/pull/26375))
-   Fix light theme select option for hashtags ([teeerevor](https://github.com/mastodon/mastodon/pull/26311))
-   Fix AVIF attachments ([c960657](https://github.com/mastodon/mastodon/pull/26264))
-   Fix incorrect URL normalization when fetching remote resources ([c960657](https://github.com/mastodon/mastodon/pull/26219), [c960657](https://github.com/mastodon/mastodon/pull/26285))
-   Fix being unable to filter posts for individual Chinese languages ([gunchleoc](https://github.com/mastodon/mastodon/pull/26066))
-   Fix preview card sometimes linking to 4xx error pages ([c960657](https://github.com/mastodon/mastodon/pull/26200))
-   Fix emoji picker button scrolling with textarea content in single-column view ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25304))
-   Fix missing border on error screen in light theme in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/26152))
-   Fix UI overlap with the loupe icon in the Explore Tab ([gol-cha](https://github.com/mastodon/mastodon/pull/26113))
-   Fix unexpected redirection to `/explore` after sign-in ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26143))
-   Fix `/api/v1/statuses/:id/unfavourite` and `/api/v1/statuses/:id/unreblog` returning non-updated counts ([c960657](https://github.com/mastodon/mastodon/pull/24365))
-   Fix clicking the “Back” button sometimes leading out of Mastodon ([c960657](https://github.com/mastodon/mastodon/pull/23953), [CSFlorin](https://github.com/mastodon/mastodon/pull/24835), [S-H-GAMELINKS](https://github.com/mastodon/mastodon/pull/24867), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25281))
-   Fix processing of `null` ActivityPub activities ([tribela](https://github.com/mastodon/mastodon/pull/26021))
-   Fix hashtag posts not being removed from home feed on hashtag unfollow ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/26028))
-   Fix for "follows you" indicator in light web UI not readable ([vmstan](https://github.com/mastodon/mastodon/pull/25993))
-   Fix incorrect line break between icon and number of reposts & favourites ([edent](https://github.com/mastodon/mastodon/pull/26004))
-   Fix sounds not being loaded from assets host ([Signez](https://github.com/mastodon/mastodon/pull/25931))
-   Fix buttons showing inconsistent styles ([teeerevor](https://github.com/mastodon/mastodon/pull/25903), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25965), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26341), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/26482))
-   Fix trend calculation working on too many items at a time ([Gargron](https://github.com/mastodon/mastodon/pull/25835))
-   Fix dropdowns being disabled for logged out users in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/25714), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25964))
-   Fix explore page being inaccessible when opted-out of trends in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/25716))
-   Fix re-activated accounts possibly getting deleted by `AccountDeletionWorker` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25711))
-   Fix `/api/v2/search` not working with following query param ([danielmbrasil](https://github.com/mastodon/mastodon/pull/25681))
-   Fix inefficient query when requesting a new confirmation email from a logged-in account ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25669))
-   Fix unnecessary concurrent calls to `/api/*/instance` in web UI ([mgmn](https://github.com/mastodon/mastodon/pull/25663))
-   Fix resolving local URL for remote content ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25637))
-   Fix search not being easily findable on smaller screens in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/25576), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25631))
-   Fix j/k keyboard shortcuts on some status lists ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25554))
-   Fix missing validation on `default_privacy` setting ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25513))
-   Fix incorrect pagination headers in `/api/v2/admin/accounts` ([danielmbrasil](https://github.com/mastodon/mastodon/pull/25477))
-   Fix non-interactive upload container being given a `button` role and tabIndex ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25462))
-   Fix always redirecting to onboarding in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/25396))
-   Fix inconsistent use of middle dot (·) instead of bullet (•) to separate items ([j-f1](https://github.com/mastodon/mastodon/pull/25248))
-   Fix spacing of middle dots in the detailed status meta section ([j-f1](https://github.com/mastodon/mastodon/pull/25247))
-   Fix prev/next buttons color in media viewer ([renchap](https://github.com/mastodon/mastodon/pull/25231))
-   Fix email addresses not being properly updated in `tootctl maintenance fix-duplicates` ([mjankowski](https://github.com/mastodon/mastodon/pull/25118))
-   Fix unicode surrogate pairs sometimes being broken in page title ([eai04191](https://github.com/mastodon/mastodon/pull/25148))
-   Fix various inefficient queries against account domains ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25126))
-   Fix video player offering to expand in a lightbox when it's in an `iframe` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25067))
-   Fix post embed previews ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25071))
-   Fix inadequate error handling in several API controllers when given invalid parameters ([danielmbrasil](https://github.com/mastodon/mastodon/pull/24947), [danielmbrasil](https://github.com/mastodon/mastodon/pull/24958), [danielmbrasil](https://github.com/mastodon/mastodon/pull/25063), [danielmbrasil](https://github.com/mastodon/mastodon/pull/25072), [danielmbrasil](https://github.com/mastodon/mastodon/pull/25386), [danielmbrasil](https://github.com/mastodon/mastodon/pull/25595))
-   Fix uncaught `ActiveRecord::StatementInvalid` in Mastodon::IpBlocksCLI ([danielmbrasil](https://github.com/mastodon/mastodon/pull/24861))
-   Fix various edge cases with local moves ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24812))
-   Fix `tootctl accounts cull` crashing when encountering a domain resolving to a private address ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23378))
-   Fix `tootctl accounts approve --number N` not aproving the N earliest registrations ([danielmbrasil](https://github.com/mastodon/mastodon/pull/24605))
-   Fix being unable to clear media description when editing posts ([c960657](https://github.com/mastodon/mastodon/pull/24720))
-   Fix unavailable translations not falling back to English ([mgmn](https://github.com/mastodon/mastodon/pull/24727))
-   Fix anonymous visitors getting a session cookie on first visit ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24584), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24650), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24664))
-   Fix cutting off first letter of hashtag links sometimes in web UI ([Gargron](https://github.com/mastodon/mastodon/pull/24623))
-   Fix crash in `tootctl accounts create --reattach --force` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24557), [danielmbrasil](https://github.com/mastodon/mastodon/pull/24680))
-   Fix characters being emojified even when using Variation Selector 15 (text) ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/20949), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24615))
-   Fix uncaught ActiveRecord::StatementInvalid exception in `Mastodon::AccountsCLI#approve` ([danielmbrasil](https://github.com/mastodon/mastodon/pull/24590))
-   Fix email confirmation skip option in `tootctl accounts modify USERNAME --email EMAIL --confirm` ([danielmbrasil](https://github.com/mastodon/mastodon/pull/24578))
-   Fix tooltip for dates without time ([c960657](https://github.com/mastodon/mastodon/pull/24244))
-   Fix missing loading spinner and loading more on scroll in Private Mentions column ([c960657](https://github.com/mastodon/mastodon/pull/24446))
-   Fix account header image missing from `/settings/profile` on narrow screens ([c960657](https://github.com/mastodon/mastodon/pull/24433))
-   Fix height of announcements not being updated when using reduced animations ([c960657](https://github.com/mastodon/mastodon/pull/24354))
-   Fix inconsistent radius in advanced interface drawer ([thislight](https://github.com/mastodon/mastodon/pull/24407))
-   Fix loading more trending posts on scroll in the advanced interface ([OmmyZhang](https://github.com/mastodon/mastodon/pull/24314))
-   Fix poll ending notification for edited polls ([c960657](https://github.com/mastodon/mastodon/pull/24311))
-   Fix max width of media in `/about` and `/privacy-policy` ([mgmn](https://github.com/mastodon/mastodon/pull/24180))
-   Fix streaming API not being usable without `DATABASE_URL` ([Gargron](https://github.com/mastodon/mastodon/pull/23960))
-   Fix external authentication not running onboarding code for new users ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23458))

##### Upgrade notes

To get the code for v4.2.0, use `git fetch && git checkout v4.2.0`.

> As always, **make sure you have backups of the database before performing any upgrades**. If you are using docker-compose, this is how a backup command might look: `docker exec mastodon_db_1 pg_dump -Fc -U postgres postgres > name_of_the_backup.dump`

##### Dependencies

External dependencies have changed since v4.1.7, with the Ruby, PostgreSQL and Node.js minimum version being higher.

-   Ruby: 3.0 to 3.2
-   PostgreSQL: 10 or newer
-   Elasticsearch (recommended, for full-text search): 7.x (OpenSearch should also work)
-   LibreTranslate (optional, for translations): 1.3.3 or newer
-   Redis: 4 or newer
-   Node: 16 or newer
-   ImageMagick: 6.9.7-7 or newer

> If your uploaded images are broken after the upgrade, it means your installed ImageMagick version is older than the new minimum version (6.9.7-7), for example if you are running Ubuntu 18.04. If this happens, you can find more information and ways to fix it [on this page](https://github.com/mastodon/mastodon/issues/25776).

##### Database replica configuration

The way Mastodon handles read replicas has changed, removing the `makara` gem and using native Rails support instead.

This changes how database replicas are configured. Instead of editing `config/database.yml`, you should use an unmodified one and use the `REPLICA_DB_NAME`, along with `REPLICA_DB_USER`, `REPLICA_DB_PASS`, `REPLICA_DB_HOST` and `REPLICA_DB_PORT`, if they differ from the primary database.

If you are using `DATABASE_URL`, you can configure your read replica in a similar way using `REPLICA_DATABASE_URL`.

##### StatsD integration

We have identified the current implementation of the StatsD integration for sidekiq to cause a lot of overhead. Therefore, we have disabled it by default, but since we do not have an alternative yet, it is still available by setting the following environment variable: `STATSD_SIDEKIQ=true`.

Please note that StatsD integration is deprecated and will not be supported in 4.3.0.

##### Streaming server changes

We have dropped built-in clustering support from the streaming server, which means, depending on the load you are facing, that you may need to run multiple instances of it and configure a load-balancer.

Unless you are using Docker, it is recommended that you update your `mastodon-streaming` unit scripts with the ones we provide:

1.  `sudo cp ~mastodon/live/dist/mastodon-streaming*.service /etc/systemd/system/`
2.  `sudo systemctl daemon-reload`
3.  `sudo systemctl restart mastodon-streaming`

If you then need to run more than one `mastodon-streaming server`, you can:

1.  Start a new instance with `sudo systemctl start mastodon-streaming@port` (e.g. `mastodon-streaming@4001`)
2.  Edit your nginx configuration file to add the new server to the load-balancing (an example is provided in the comments in `dist/nginx.conf`)

##### Automatic update checking

Starting from this release, Mastodon will periodically check for updates by querying `https://api.joinmastodon.org/update-check` every 30 minutes in a background job.
That URL can be changed using the `UPDATE_CHECK_URL` environment variable, and the feature outright disabled by setting that variable to an empty string (`UPDATE_CHECK_URL=`).

##### Update steps

The following instructions are for updating from 4.1.9.

If you are upgrading directly from an earlier release, please carefully read the upgrade notes for the skipped releases as well, as they often require extra steps such as database migrations.

**Non-Docker only:**

0.  If you are using `rbenv`, [update the list of available versions](https://github.com/rbenv/rbenv#updating-the-list-of-available-ruby-versions) and install Ruby 3.2.2 by doing `RUBY_CONFIGURE_OPTS=--with-jemalloc rbenv install` in the Mastodon install directory (e.g. `/home/mastodon/live`)
1.  Install dependencies: `bundle install` and `yarn install --frozen-lockfile`
2.  Precompile the assets: `RAILS_ENV=production bundle exec rails assets:precompile`
3.  Run the *pre-deployment* database migrations by specifying the `SKIP_POST_DEPLOYMENT_MIGRATIONS=true` environment variable: `SKIP_POST_DEPLOYMENT_MIGRATIONS=true RAILS_ENV=production bundle exec rails db:migrate`
4.  Restart all Mastodon processes
5.  Run the *post-deployment* database migrations: `RAILS_ENV=production bundle exec rails db:migrate`
6.  If you use Elasticsearch, rebuild the search indexes with `RAILS_ENV=production bin/tootctl search deploy --reset-chewy`

**Using Docker:**

1.  Run the *pre-deployment* database migrations by specifying the `SKIP_POST_DEPLOYMENT_MIGRATIONS=true` environment variable: `docker-compose run --rm -e SKIP_POST_DEPLOYMENT_MIGRATIONS=true web bundle exec rails db:migrate`
2.  Restart all Mastodon processes
3.  Run the *post-deployment* database migrations: `docker-compose run --rm web bundle exec rails db:migrate`
4.  If you use Elasticsearch, rebuild the search indexes with `docker-compose run --rm web bin/tootctl search deploy --reset-chewy`

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi45OS4wIiwidXBkYXRlZEluVmVyIjoiMzYuOTkuMCIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->

Reviewed-on: https://git.home/nrdufour/home-ops/pulls/103
Co-authored-by: Renovate <renovate@ptinem.io>
Co-committed-by: Renovate <renovate@ptinem.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants