How to model local exceptions to distributed tables

All we need is an easy explanation of the problem, so here it is.

I am looking for possible ways to model a distributed table(s) and provide local exceptions to that table. Consider a situation where head office publishes a list of products/prices that is distributed to branches and they load this price list into their local database. Now sometimes these branches want to change the head office price to a local price. Clearly we can simply update the price in the local database, but then it gets lost when the next head office update is applied. We also only want to block that single column(s), not changes to other fields in the row.

EG. Head office send

| Name | Price |
|------|-------|
| ABC  | $5    |
| DEF  | $8    |

In one database (out of say 100) we want ABC to have a price of $3. And this $3 should continue to apply even as future updates from head office are received. Head office are not aware of this local exeception.

In the past we’ve done this in two ways. The first was to maintain a second LocalProducts table with the changes to be applied – then when the head office version was applied, any local changes would then be read from LocalProducts and replace the distributed value.
This works OK, but suffers as the table sizes go up.

The second method we tried was to duplicate the column and use coalesce
eg rather than select price where name='abc' we would use select coalesce(localprice,price) where name='abc' This works ok, but had the overhead that we had to change every SQL and remember to do it on future development too.

Are there any other better methods? I am looking for a database agnostic solution, the target databases are ODBC (90% sql/server), SQL-Lite or MS-Access. Access can be ignored if it helps, it is being actively phased out. Engine specific solutions are ok, we can detect the engine and use a different technique, but would prefer not to have a different technique for each engine.

Other notes

  • The data is distributed by the application, not built in replication. This means we can have code/logic during the loading phase
  • Multiple tables are require this feature, not just one as the example above
  • Some of the tables have a reasonable number of columns that could be involved, but typically not.
  • Most of the tables aren’t large, perhaps 50M rows in the biggest, and only 7 rows in the smallest case.

(please feel free to retag this question – not exactly sure which tags are relevant. And yes I have spent many hours over the years investigating this)

How to solve :

I know you bored from this bug, So we are here to help you! Take a deep breath and look at the explanation of your problem. We have many solutions to this problem, But we recommend you to use the first method because it is tested & true method that will 100% work for you.

Method 1

Put the COALESCE in a view write queries against the view, not the base tables. It would help if you could remove access to these tables and grant access to the view only.

There will be a small runtime penalty with this, as two tables must be read. If you could change the load to perform the COALESCE once at load time that would be better. Subsequent changes to local price must be written to both local table and to the master list. It would not be possible to run reports on, say, the cost of local pricing as the official prices would be unknown.

Note: Use and implement method 1 because this method fully tested our system.
Thank you 🙂

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply