v1.5.1

[ruslandoga]

The highlights of this release are:

• City and region level geolocation (disabled by default, please see below for the guide)
• Google Analytics integration: https://plausible.io/docs/google-analytics-import (Setup guide)
• CSV data export: https://plausible.io/docs/export-stats
• ARM64 support
• Improved first launch experience

New first launch

Admin user registration is now done through the UI instead of configuration variables. At first launch, users are presented with a registration page which will create the admin account. Admin account related configuration variables are ignored going forward. See below for details.

Admin user registration is required from 1.5 onwards. We are deprecating the DISABLE_AUTH option which allowed Plausible to be used without any user account at all.

Users with existing admin accounts are not affected, but we recommend removing the unused configuration variables. A benefit of moving admin account management to the UI is that you can now change the admin email account through the UI. Previously, this was done by manual database changes and configuration change which required restarting the service.

Upgrading Plausible Analytics to v1.5

In your docker-compose.yml update the image used for plausible to v1.5.1

plausible:
- image: plausible/analytics:v1.4
+ image: plausible/analytics:v1.5.1

_{Please make sure to not use v1.5.0 release which had startup problems: #2491}

Restart the container

$ docker compose stop plausible
$ docker compose rm plausible
$ docker compose up -d

This will boot up the new version of the app.

City-level geolocation setup (Optional, requires more memory)

⚠️ Note that this city level geolocation feature requires extra ~1GB of RAM to work. If you don't have enough RAM available, you might experience slow container startups and restarts.

City level geolocation can be enabled by mounting a volume with the downloaded geonames.csv and geolite2-city.mmdb and setting GEONAMES_SOURCE_FILE and IP_GEOLOCATION_DB to point to them:

$ # custom (smaller) csv can be used, see https://github.com/plausible/location#cities for more info
$ curl -O https://s3.eu-central-1.wasabisys.com/plausible-application/geonames.csv

$ export MAXMIND_LICENSE_KEY=LNpsJCCKPis6XvBP # this key is not valid anymore
$ curl -L "https://download.maxmind.com/app/geoip_download?edition_id=GeoLite2-City&license_key=${MAXMIND_LICENSE_KEY}&suffix=tar.gz" -o geolite2-city.mmdb.gz
$ gunzip geolite2-city.mmdb.gz

docker-compose.yml

  plausible:
    image: plausible/analytics:v1.5.1
+   volumes:
+     - ./geonames.csv:/etc/app/geonames.csv:ro
+     - ./geolite2-city.mmdb:/etc/app/geolite2-city.mmdb:ro

plausible-conf.env

BASE_URL=replace-me
SECRET_KEY_BASE=replace-me
+ GEONAMES_SOURCE_FILE=/etc/app/geonames.csv
+ IP_GEOLOCATION_DB=/etc/app/geolite2-city.mmdb

Upgrading ClickHouse to 22.6 (Optional, but recommended)

In your docker-compose.yml update the image used for plausible_events_db to a newer ClickHouse version:

plausible_events_db:
- image: yandex/clickhouse-server:21.3.2.5
+ image: clickhouse/clickhouse-server:22.6-alpine

Restart the container

$ docker compose stop plausible plausible_events_db
$ docker compose rm plausible_events_db
$ docker compose up -d

This will boot up the new version of ClickHouse.

Related PR: plausible/community-edition#45

Upgrading PostgreSQL to 14 (Optional, but recommended)

Upgrading PostgreSQL is a bit trickier, here's the upgrade guide.

Removing deprecated commands and environment variables (Optional, but recommended)

ADMIN_USER_EMAIL, ADMIN_USER_NAME, ADMIN_USER_PWD, DISABLE_AUTH environment variables are no longer supported and are ignored if set.

Please remove them from plausible-conf.env:

- ADMIN_USER_EMAIL=replace-me
- ADMIN_USER_NAME=replace-me
- ADMIN_USER_PWD=replace-me
- DISABLE_AUTH=true
  BASE_URL=replace-me
  SECRET_KEY_BASE=replace-me

/entrypoint.sh db init-admin command that was responsible for creating the admin user is also deprecated.

Please remove it from docker-compose.yml:

plausible:
  image: plausible/analytics:v1.5.1
- command: sh -c "sleep 10 && /entrypoint.sh db createdb && /entrypoint.sh db migrate && /entrypoint.sh db init-admin && /entrypoint.sh run"
+ command: sh -c "sleep 10 && /entrypoint.sh db createdb && /entrypoint.sh db migrate && /entrypoint.sh run"

Changelog

Following changes have been made since v1.4:

Added

• Set a different interval on the top graph Adds ability to select graph detail (interval) #1574 (thanks to @Vigasaurus for this feature)
• A tagged-events script extension for out-of-the-box custom event tracking
• The ability to escape | characters with \ in Stats API filter values
• An upper bound of 1000 to the limit parameter in Stats API
• The exclusions script extension now also takes a data-include attribute tag
• A file-downloads script extension for automatically tracking file downloads as custom events
• Integration with Matomo's referrer spam list to block known spammers
• API route PUT /api/v1/sites/goals with form params site_id, event_name and/or page_path, and goal_type with supported types event and page
• API route DELETE /api/v1/sites/goals/:goal_id with form params site_id
• The public breakdown endpoint can be queried with the "events" metric
• Data exported via the download button will contain CSV data for all visible graps in a zip file.
• Region and city-level geolocation City level location data #1449
• The u option can now be used in the manual extension to specify a URL when triggering events.
• Delete a site and all related data through the Sites API
• Subscribed users can see their Paddle invoices from the last 12 months under the user settings
• Allow custom styles to be passed to embedded iframe Allow custom styles to be passed to embedded iframe #1522
• New UTM Tags utm_content and utm_term Support for utm_term and utm_content #515
• If a session was started without a screen_size it is updated if an event with screen_size occurs
• Added LISTEN_IP configuration parameter no pg_hba.conf entry for host when connecting to Postgres via Heroku #1189
• The breakdown endpoint with the property query property=event:goal returns custom goal properties (within props)
• Added IPv6 Ecto support (via the environment-variable ECTO_IPV6)
• New filter type: contains, available for page, entry_page, exit_page
• Add filter for custom property
• Add ability to import historical data from GA: [Continued] Google Analytics import #1753
• API route GET /api/v1/sites/:site_id
• Hovering on top of list items will now show a tooltip with the exact number instead of a shortened version
• Filter goals in realtime filter by clicking goal name
• The time format (12 hour or 24 hour) for graph timelines is now presented based on the browser's defined language
• Choice of metric for main-graph both in UI and API (visitors, pageviews, bounce_rate, visit_duration) Adds Main Graph Metric Selection #1364
• New width=manual mode for embedded dashboards Embed improvements #2148
• Add more timezone options
• Add new strategy to recommend timezone when creating a new site
• Alert outgrown enterprise users of their usage Manually lock outgrown enterprise accounts #2197
• Manually lock and unlock enterprise users Manually lock outgrown enterprise accounts #2197
• ARM64 support for docker images ARM64 support for docker images #2103
• Add support for international domain names (IDNs) International Domain Names (IDN) Support #2034
• Allow self-hosters to register an account on first launch
• Fix ownership transfer invitation link in self-hosted deployments

Fixed

• Plausible script does not prevent default if it's been prevented by an external script plausible/analytics#1941
• Hash part of the URL can now be used when excluding pages with script.exclusions.hash.js.
• UI fix where multi-line text in pills would not be underlined properly on small screens.
• UI fix to align footer columns
• Guests can now use the favicon to toggle additional info about the site bing viewed (such as in public embeds).
• Fix SecurityError in tracking script when user has blocked all local storage
• Prevent dashboard graph from being selected when long pressing on the graph in a mobile browser
• The exported pages.csv file now includes pageviews again plausible/analytics#1878
• Fix a bug where city, region and country filters were filtering stats but not the location list
• Fix a bug where regions were not being saved
• Timezone offset labels now update with time changes
• Render 404 if shared link auth cannot be verified plausible/analytics#2225
• Restore compatibility with older format of shared links plausible/analytics#2225
• Fix 'All time' period for sites with no recorded stats plausible/analytics#2277
• Ensure settings page can be rendered after a form error plausible/analytics#2278
• Ensure newlines from settings files are trimmed plausible/analytics#2480

Changed

• script.file-downloads.outbound-links.js only sends an outbound link event when an outbound download link is clicked
• Plausible script now uses callback navigation (instead of waiting for 150ms every time) when sending custom events
• Cache the tracking script for 24 hours
• Move entry_page and exit_page to be part of the Page filter group
• Paginate /api/sites results and add a View all link to the site-switcher dropdown in the dashboard.
• Remove the + Add Site link to the site-switcher dropdown in the dashboard.
• DISABLE_REGISTRATIONS configuration parameter can now accept invite_only to allow invited users to register an account while keeping regular registrations disabled Adds "invite_only" disable_registration config option #1841
• New and improved Session tracking module for higher throughput and lower latency. PR#1934
• Do not display ZZ country code in countries report PR#1934
• Add fallback icon for when DDG favicon cannot be fetched PR#2279

Security

• Add Content-Security-Policy header to favicon path

Deprecated

• DISABLE_AUTH is no longer supported

Release candidates

• v1.5.0-rc.0
• v1.5.0-rc.1

---

This discussion was created from the release v1.5.1.

[nmaggioni]

When specifying the GEONAMES_SOURCE_FILE env var and feeding it with the file specified in the release notes (https://s3.eu-central-1.wasabisys.com/plausible-application/geonames.csv), Plausible crashes with the following log:

Loading plausible..
Starting dependencies..
Starting repos..
create Plausible.Repo database if it doesn't exist
create Plausible.ClickhouseRepo database if it doesn't exist
Creation of Db successful!
Loading plausible..
Starting dependencies..
Starting repos..
Running migrations for Elixir.Plausible.Repo
Running migrations for Elixir.Plausible.ClickhouseRepo
Migrations successful!
{"Kernel pid terminated",application_controller,"{application_start_failure,plausible,{bad_return,{{'Elixir.Plausible.Application',start,[normal,[]]},{'EXIT',{timeout,{'Elixir.Task',await,[#{'__struct__' => 'Elixir.Task',owner => <0.2950.0>,pid => <0.2954.0>,ref => #Ref<0.132244393.3611099137.47192>},30000]}}}}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,plausible,{bad_return,{{'Elixir.Plausible.Application',start,[normal,[]]},{'EXIT',{timeout,{'Elixir.Task',await,[#{'__struct__' => 'Elixir.Task',owner => <0.2950.0>,pid => <0.2954.0>,ref => #Ref<0.132244393.3611099137.47192>},30000]}}}}}})

Crash dump is being written to: erl_crash.dump...

RAM limits are not in place in this deployment, and the crashdump does not indicate troubles with OOM errors (~500MB used).

When generating the city mappings manually through your package the result is the same as when using the prebuilt ones, but using the lite variant (geonames.lite.csv) seems to allow Plausible to start and work correctly.

[nmaggioni]

Thank you, hidden replies masked that bit of info.

[IzMichael]

More info and a possible workaround (requires container image to be rebuilt)

Is there a way to fix this without rebuilding a container image? Is this planned for a future version of the plausible/analytics image? Thanks!

[ruslandoga]

Right now the timeout value (30s) is not possible to change without a rebuild. Not sure if it's going to change in the future version. Ideally we'd find a way to make it fast for everyone, without the need to increase the timeout value.

[joshavant]

Confirming I'm also seeing this problem. This is a difficult default behavior.

[thiboot]

i'm having the exact same problem too

[joshavant]

If I use the maxmindinc/geoipupdate Docker image and have it re-download the Maxmind DB every few days, will this release notice the updated DB file in the volume and hot reload the Maxmind DB?

(I know there was a conversation about this somewhere around the repo recently, but I can't seem to find it and check if that made it in...)

[joshavant]

Aha! Found the comment:
#2324 (reply in thread)

[jesperordrup]

Invite_only doesnt seem to work

#1841 (comment)

[ruslandoga]

👋

v2.0.0-rc.0 pre-release is now available!

[B0biN]

Hello,
I'm trying to upgrade from analytics:v1.4 to analytics:v1.5.1 and there is error from plausible container:

plausible_1  | Loading plausible..
plausible_1  | Starting dependencies..
plausible_1  | Starting repos..
plausible_1  | create Plausible.Repo database if it doesn't exist
plausible_1  | create Plausible.ClickhouseRepo database if it doesn't exist
plausible_1  | Creation of Db successful!
plausible_1  | Loading plausible..
plausible_1  | Starting dependencies..
plausible_1  | Starting repos..
plausible_1  | Running migrations for Elixir.Plausible.Repo
plausible_1  | Running migrations for Elixir.Plausible.ClickhouseRepo
plausible_1  | ** (Clickhousex.Error) Code: 10. DB::Exception: Wrong column name. Cannot find column `entry.meta` to drop. (NOT_FOUND_COLUMN_IN_BLOCK) (version 22.6.9.11 (official build))
plausible_1  |
plausible_1  |     (ecto_sql 3.8.3) lib/ecto/adapters/sql.ex:932: Ecto.Adapters.SQL.raise_sql_call_error/1
plausible_1  |     (elixir 1.13.4) lib/enum.ex:1593: Enum."-map/2-lists^map/1-0-"/2
plausible_1  |     (ecto_sql 3.8.3) lib/ecto/adapters/sql.ex:1024: Ecto.Adapters.SQL.execute_ddl/4
plausible_1  |     (ecto_sql 3.8.3) lib/ecto/migration/runner.ex:352: Ecto.Migration.Runner.log_and_execute_ddl/3
plausible_1  |     (ecto_sql 3.8.3) lib/ecto/migration/runner.ex:117: anonymous fn/6 in Ecto.Migration.Runner.flush/0
plausible_1  |     (elixir 1.13.4) lib/enum.ex:2396: Enum."-reduce/3-lists^foldl/2-0-"/3
plausible_1  |     (ecto_sql 3.8.3) lib/ecto/migration/runner.ex:116: Ecto.Migration.Runner.flush/0
plausible_1  |     (stdlib 3.17.1) timer.erl:166: :timer.tc/1

The same issue upgrade from analytics:v1.4 to analytics:v2.0.0.

[ruslandoga]

@B0biN 👋

There has been a similar issue: #2324 (comment)

Guess we'd need to look into why it happens...

[ruslandoga]

What ClickHouse version are you using currently? For some reason the previous migration that creates column entry.meta

analytics/priv/ingest_repo/migrations/20220404123000_add_entry_props_to_session.exs

Line 6 in 8c870f7

add :"entry.meta", :"Nested(key String, value String)"

silently fails somehow. Or maybe there is an issue with tracking executed migrations and 20220404123000 is marked as "executed" without actually being executed.

[B0biN]

@ruslandoga
Now I have 22.6-alpine and previously was 21.3.2.5.

I was migrated ClicHouse by the:

plausible_events_db:
- image: yandex/clickhouse-server:21.3.2.5
+ image: clickhouse/clickhouse-server:22.6-alpine

... but issue with Cannot find column `entry.meta` to drop. has been in 21.3.2.5 too.