Suggestion for nearly gap-less sequences in postgres

All we need is an easy explanation of the problem, so here it is.

In PostgreSQL sequences are designed to have gaps for reasons mentioned in this post https://stackoverflow.com/questions/9984196/postgresql-gapless-sequences. My question is: could sequences not be made to be nearly gap-less by periodically (let’s say once a day) checking which sequence numbers are unused and adding those back to the sequence to use?

I assume that currently postgres implements a sequence as a counter which it increments every time it’s accessed. Instead it could be implemented with a list of free values & a counter. The sequence would then work as follows:

# python style pseudo code

class Sequence:
    def __init__(self):
        self.free_numbers_set = []
        self.counter = 0

    def get_next_sequence_value(self):
        if not isempty(self.free_numbers_set):
            id = self.free_numbers_set.pop(0)
        else:
            id = self.counter
            id ++

        return id

    def add_free_numbers(self, column):
        for i in range(0:self.counter):
            if not i in column:
                free_numbers_set.append(i)
        free_numbers_set.sort()

Everytime we need a new id we run get_next_sequence_value() and periodically we run add_free_numbers(column)

Updates addressing some of the questions

  1. For identity columns, getting the currently used values would be trivial. In other cases it would be up to the user to supply a list of values which are already in use.
  2. The interaction with MVCC would need to be thought about. One could potentially implement it such that during the missing values update, the sequence would default to only incrementing the counter – this would avoid conflicts.

I believe this solution could provide less sparse sequences which would allow users to use smaller id types (int vs bigint) while being backwards compatible with the current implementation.

How to solve :

I know you bored from this bug, So we are here to help you! Take a deep breath and look at the explanation of your problem. We have many solutions to this problem, But we recommend you to use the first method because it is tested & true method that will 100% work for you.

Method 1

The problem is that it is impossible to determine how and where the sequence is used. It doesn’t have to be the DEFAULT clause of a column that owns the sequence: nextval could be called anywhere and used in creative ways, even outside the database.

Even if you consider only the case of an identity column, scanning the table for "missing values" would be quite expensive, and it is not clear how that should interact with MVCC.

Finally, most of the requests for gap-less sequences I see are asking for monotonically increasing values, which this solution would not provide.

Note: Use and implement method 1 because this method fully tested our system.
Thank you 🙂

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply