Failed to run Tarantool replication

All we need is an easy explanation of the problem, so here it is.

I’m trying to reproduce the tarantool master-replication from documentation

so I made master/master.lua, rep1/rep1.lua, rep2/rep2.lua, cd for each directory and run it, but it shows something like this (for 10+ minutes, no progress):

@ /tmp/master
└─[0] <> tarantool master.lua
2021-05-21 21:12:43.941 [884153] main/103/master.lua C> Tarantool 2.7.2-14-ga308ba8e2
2021-05-21 21:12:43.941 [884153] main/103/master.lua C> log level 5
2021-05-21 21:12:43.941 [884153] main/103/master.lua I> wal/engine cleanup is paused
2021-05-21 21:12:43.941 [884153] main/103/master.lua I> mapping 268435456 bytes for memtx tuple arena...
2021-05-21 21:12:43.941 [884153] main/103/master.lua I> Actual slab_alloc_factor calculated on the basis of desired slab_alloc_factor = 1.044274
2021-05-21 21:12:43.941 [884153] main/103/master.lua I> mapping 134217728 bytes for vinyl tuple arena...
2021-05-21 21:12:43.942 [884153] main/103/master.lua I> instance uuid b78ffe3e-d8d3-4f98-b213-dae4ac0b6858
2021-05-21 21:12:43.942 [884153] iproto/101/main I> binary: bound to 0.0.0.0:13301
2021-05-21 21:12:43.942 [884153] main/103/master.lua I> connecting to 2 replicas
2021-05-21 21:12:43.942 [884153] main/111/applier/[email protected]:33301 I> can't connect to master
2021-05-21 21:12:43.942 [884153] main/111/applier/[email protected]:33301 coio.cc:108 !> SystemError connect, called on fd 17, aka 127.0.0.1:59352: Connection refused
2021-05-21 21:12:43.942 [884153] main/111/applier/[email protected]:33301 I> will retry every 1.00 second
2021-05-21 21:12:43.942 [884153] main/110/applier/[email protected]:23301 I> can't connect to master
2021-05-21 21:12:43.942 [884153] main/110/applier/[email protected]:23301 coio.cc:108 !> SystemError connect, called on fd 16, aka 127.0.0.1:49188: Connection refused
2021-05-21 21:12:43.942 [884153] main/110/applier/[email protected]:23301 I> will retry every 1.00 second
2021-05-21 21:12:45.944 [884153] main/110/applier/[email protected]:23301 I> remote master e242f602-6591-4eb5-9194-f64d7541ea67 at 127.0.0.1:23301 running Tarantool 2.7.2
2021-05-21 21:12:47.945 [884153] main/111/applier/[email protected]:33301 I> remote master 7d21d786-f0fa-440c-8b19-4c2f66fe47fe at 127.0.0.1:33301 running Tarantool 2.7.2
2021-05-21 21:12:47.945 [884153] main/103/master.lua I> connected to 2 replicas
2021-05-21 21:12:47.945 [884153] main/111/applier/[email protected]:33301 I> failed to authenticate
2021-05-21 21:12:47.945 [884153] main/111/applier/[email protected]:33301 box.cc:2381 E> ER_LOADING: Instance bootstrap hasn't finished yet
2021-05-21 21:12:47.945 [884153] main/111/applier/[email protected]:33301 I> will retry every 1.00 second

@ /tmp/rep1
└─[0] <> tarantool rep1.lua
2021-05-21 21:12:45.793 [884163] main/103/rep1.lua C> Tarantool 2.7.2-14-ga308ba8e2
2021-05-21 21:12:45.793 [884163] main/103/rep1.lua C> log level 5
2021-05-21 21:12:45.793 [884163] main/103/rep1.lua I> wal/engine cleanup is paused
2021-05-21 21:12:45.793 [884163] main/103/rep1.lua I> mapping 268435456 bytes for memtx tuple arena...
2021-05-21 21:12:45.793 [884163] main/103/rep1.lua I> Actual slab_alloc_factor calculated on the basis of desired slab_alloc_factor = 1.044274
2021-05-21 21:12:45.793 [884163] main/103/rep1.lua I> mapping 134217728 bytes for vinyl tuple arena...
2021-05-21 21:12:45.793 [884163] main/103/rep1.lua I> instance uuid e242f602-6591-4eb5-9194-f64d7541ea67
2021-05-21 21:12:45.793 [884163] iproto/101/main I> binary: bound to 0.0.0.0:23301
2021-05-21 21:12:45.793 [884163] main/103/rep1.lua I> connecting to 2 replicas
2021-05-21 21:12:45.794 [884163] main/111/applier/[email protected]:33301 I> can't connect to master
2021-05-21 21:12:45.794 [884163] main/111/applier/[email protected]0.0.1:33301 coio.cc:108 !> SystemError connect, called on fd 17, aka 127.0.0.1:59388: Connection refused
2021-05-21 21:12:45.794 [884163] main/111/applier/[email protected]:33301 I> will retry every 1.00 second
2021-05-21 21:12:45.794 [884163] main/110/applier/[email protected]:13301 I> remote master b78ffe3e-d8d3-4f98-b213-dae4ac0b6858 at 127.0.0.1:13301 running Tarantool 2.7.2
2021-05-21 21:12:47.796 [884163] main/111/applier/[email protected]:33301 I> remote master 7d21d786-f0fa-440c-8b19-4c2f66fe47fe at 127.0.0.1:33301 running Tarantool 2.7.2
2021-05-21 21:12:47.796 [884163] main/103/rep1.lua I> connected to 2 replicas
2021-05-21 21:12:47.796 [884163] main/110/applier/[email protected]:13301 I> failed to authenticate
2021-05-21 21:12:47.796 [884163] main/110/applier/[email protected]:13301 box.cc:2381 E> ER_LOADING: Instance bootstrap hasn't finished yet
2021-05-21 21:12:47.796 [884163] main/110/applier/[email protected]:13301 I> will retry every 1.00 second

@ /tmp/rep2
└─[0] <> tarantool rep2.lua
2021-05-21 21:12:47.435 [884173] main/103/rep2.lua C> Tarantool 2.7.2-14-ga308ba8e2
2021-05-21 21:12:47.435 [884173] main/103/rep2.lua C> log level 5
2021-05-21 21:12:47.435 [884173] main/103/rep2.lua I> wal/engine cleanup is paused
2021-05-21 21:12:47.435 [884173] main/103/rep2.lua I> mapping 268435456 bytes for memtx tuple arena...
2021-05-21 21:12:47.435 [884173] main/103/rep2.lua I> Actual slab_alloc_factor calculated on the basis of desired slab_alloc_factor = 1.044274
2021-05-21 21:12:47.435 [884173] main/103/rep2.lua I> mapping 134217728 bytes for vinyl tuple arena...
2021-05-21 21:12:47.436 [884173] main/103/rep2.lua I> instance uuid 7d21d786-f0fa-440c-8b19-4c2f66fe47fe
2021-05-21 21:12:47.436 [884173] iproto/101/main I> binary: bound to 0.0.0.0:33301
2021-05-21 21:12:47.436 [884173] main/103/rep2.lua I> connecting to 2 replicas
2021-05-21 21:12:47.437 [884173] main/110/applier/[email protected]:13301 I> remote master b78ffe3e-d8d3-4f98-b213-dae4ac0b6858 at 127.0.0.1:13301 running Tarantool 2.7.2
2021-05-21 21:12:47.437 [884173] main/111/applier/[email protected]:23301 I> remote master e242f602-6591-4eb5-9194-f64d7541ea67 at 127.0.0.1:23301 running Tarantool 2.7.2
2021-05-21 21:12:47.437 [884173] main/103/rep2.lua I> connected to 2 replicas
2021-05-21 21:12:47.437 [884173] main/110/applier/[email protected]:13301 I> failed to authenticate
2021-05-21 21:12:47.437 [884173] main/110/applier/[email protected]:13301 box.cc:2381 E> ER_LOADING: Instance bootstrap hasn't finished yet
2021-05-21 21:12:47.437 [884173] main/110/applier/[email protected]:13301 I> will retry every 1.00 second

the code are here:

master.lua

box.cfg{
  listen = 13301,
        replication = {'replicator:[email protected]:23301',  -- master URI
                 'replicator:[email protected]:33301'}, -- replica URI
  read_only = false
}
box.once("schema", function()
   box.schema.user.create('replicator', {password = 'password'})
   box.schema.user.grant('replicator', 'replication') -- grant replication role
   box.schema.space.create("test")
   box.space.test:create_index("primary")
   print('box.once executed on master')
end)

rep1.lua

box.cfg{
  listen = 23301,
  replication = {'replicator:[email protected]:13301',  -- master URI
                 'replicator:[email protected]:33301'}, -- replica URI
  read_only = true
}
box.once("schema", function()
   box.schema.user.create('replicator', {password = 'password'})
   box.schema.user.grant('replicator', 'replication') -- grant replication role
   box.schema.space.create("test")
   box.space.test:create_index("primary")
   print('box.once executed on replica 1')
end)

rep2.lua

box.cfg{
  listen = 33301,
  replication = {'replicator:[email protected]:13301',  -- master URI
                 'replicator:[email protected]:23301'}, -- replica URI
  read_only = true
}
box.once("schema", function()
   box.schema.user.create('replicator', {password = 'password'})
   box.schema.user.grant('replicator', 'replication') -- grant replication role
   box.schema.space.create("test")
   box.space.test:create_index("primary")
   print('box.once executed on replica 2')
end)

What’s wrong with it?

How to solve :

I know you bored from this bug, So we are here to help you! Take a deep breath and look at the explanation of your problem. We have many solutions to this problem, But we recommend you to use the first method because it is tested & true method that will 100% work for you.

Method 1

Nevermind, my bad, first replication line should be the master, so it should be something like this:

        replication = {'replicator:[email protected]:13301',  -- master URI
                 'replicator:[email protected]:23301', -- replica URI
                 'replicator:[email protected]:33301'}, -- replica URI

Note: Use and implement method 1 because this method fully tested our system.
Thank you 🙂

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply