Autovacuum does not clear the database

All we need is an easy explanation of the problem, so here it is.

Autovacuum does not clear the database.
Databases middle-db
Postgres 10.18, AWS RDS(vCPU 2, RAM 8Gb, SSD(gp2) 1100Gib)

Table "spree_datafeed_products"

relid               | 16556
schemaname          | public
relname             | spree_datafeed_products
seq_scan            | 20
seq_tup_read        | 365522436
idx_scan            | 962072108
idx_tup_fetch       | 9929276855
n_tup_ins           | 2846455
n_tup_upd           | 35778058
n_tup_del           | 284291955
n_tup_hot_upd       | 0
n_live_tup          | 3546840
n_dead_tup          | 338790851
n_mod_since_analyze | 307930753
last_vacuum         | 
last_autovacuum     | 
last_analyze        | 
last_autoanalyze    | 2022-04-29 13:01:43.985749+00
vacuum_count        | 0
autovacuum_count    | 0
analyze_count       | 0
autoanalyze_count   | 1

Table and indexes sizes:

indexname                           | size  
index_spree_datafeed_products_on_updated_at                  | 48 GB
index_spree_datafeed_products_on_state                       | 35 GB
index_spree_datafeed_products_on_size_variant_field          | 40 GB
index_spree_datafeed_products_on_product_id                  | 32 GB
index_spree_datafeed_products_on_original_id                 | 31 GB
index_spree_datafeed_products_on_datafeed_id                 | 42 GB
index_spree_datafeed_products_on_datafeed_id_and_original_id | 31 GB
index_spree_datafeed_products_on_data_hash                   | 39 GB
spree_datafeed_products_pkey                                 | 18 GB

 pg_size_pretty  - 419 GB

Worker:

datid            | 16404
datname          | milanstyle_production
pid              | 2274
backend_start    | 2022-05-01 19:52:00.066097+00
xact_start       | 2022-05-01 19:52:00.23692+00
query_start      | 2022-05-01 19:52:00.23692+00
state_change     | 2022-05-01 19:52:00.236921+00
wait_event_type  | 
wait_event       | 
state            | active
backend_xid      | 
backend_xmin     | 1301636863
query            | autovacuum: VACUUM ANALYZE public.spree_datafeed_products
backend_type     | autovacuum worker

Settings:

autovacuum on
autovacuum_analyze_scale_factor 0.05
autovacuum_analyze_threshold 50
autovacuum_freeze_max_age 200000000
autovacuum_max_workers 3
autovacuum_multixact_freeze_max_age 400000000
autovacuum_naptime 30
autovacuum_vacuum_cost_delay 20
autovacuum_vacuum_cost_limit -1
autovacuum_vacuum_scale_factor 0.1
autovacuum_vacuum_threshold 50

The garbage cleaning script has accumulated a lot of deleted entries. We have been waiting for more than a week (autoclearance). What is the problem? Why is the database failing?

How to solve :

I know you bored from this bug, So we are here to help you! Take a deep breath and look at the explanation of your problem. We have many solutions to this problem, But we recommend you to use the first method because it is tested & true method that will 100% work for you.

Method 1

It will take a very long time to vacuum a table of that size with the those settings (assuming the ones you don’t show are at their defaults).

If the live tuples are really 1% of the dead tuples, the easiest way out of this hole might be a VACUUM FULL of the table.

To avoid getting back into that hole again, you should at least drop autovacuum_vacuum_cost_delay to 2 and increase autovacuum_work_mem to at least 256MB (but I would probably do 1GB).

I would also look through the logs to see if autovacs have a history of getting cancelled before finishing.

Note: Use and implement method 1 because this method fully tested our system.
Thank you 🙂

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply