Thursday, 27 September 2007

Setting up Slony-I with pgAdmin

pgAdmin has had the ability to create and manage Slony-I replication clusters for some time, however it is designed to allow the user to work directly with the lower-level Slony concepts such as listens and paths. This is pretty flexible, but isn't particularly user-friendly. We are hoping to include some wizards in the future to make it much easier to setup and modify clusters, but in the meantime here's a long overdue walkthrough using the existing features to create a 3 node replication cluster.

In this example, a master server is setup with two direct slaves. This example was written and tested using Slony-I v1.2.11 and PostgreSQL 8.2.5, running on a single Windows XP machine. The PostgreSQL pgbench utility is used to generate the test schema and workload.

1) Create 3 databases, master, slave1 and slave2 and ensure pl/pgsql is setup in each.

2) Create a pgbench schema in the master database:
> pgbench -i -U postgres master
3) Add a primary key called history_pkey to the history table on the tid, bid and aid columns.

4) Create a schema-only dump of the master database, and load it into slave1 and slave2:
> pg_dump -s -U postgres master > schema.sql
> psql -U postgres slave1 < psql -U postgres slave2
5) Create Slony config files for each slon engine (daemon on *nix). The files should contain just the following two lines:
conn_info='host= port=5432 user=postgres dbname=master'
Create a file for each database, adjusting the dbname parameter as required and adding any other connection options that may be needed. (Windows only) Install the Slony-I service: > slon -regservice Slony-I

6) Register each of the engines (this is only necessary on Windows - on *nix the slon daemons may be started individually and given the path to the config file on the command line using the -f option):
> slon -addengine Slony-I C:\slony\master.conf
> slon -addengine Slony-I C:\slony\slave1.conf
> slon -addengine Slony-I C:\slony\slave2.conf
7) In pgAdmin under the Replication node in the master database, create a new Slony-I cluster using the following options:
Join existing cluster: Unchecked
Cluster name: pgbench
Local node: 1 Master node
Admin node: 99 Admin node
8) Under the Replication node, create a Slony-I cluster in each of the slave databases using the following options:
Join existing cluster: Checked
Server: <Select the server containing the master database>
Database: master
Cluster name: pgbench
Local node: 10 Slave node 1
Admin node: 99 - Admin node
Join existing cluster: Checked
Server: <Select the server containing the master database>
Database: master
Cluster name: pgbench
Local node: 20 Slave node 2
Admin node: 99 - Admin node
9) Create Paths on the master to both slaves, and on each slave back to the master. Create the paths under each node on the master, using the connection strings specified in the slon config files. Note that future restructuring of the cluster may require additional paths to be defined.

10) Create a Replication Set on the master using the following settings:
ID: 1
Comment: pgbench set
11) Add the tables to the replication set using the following settings:
Table: public.accounts
ID: 1
Index: accounts_pkey

Table: public.branches
ID: 2
Index: branches_pkey

Table: public.history
ID: 3
Index: history_pkey

Table: public.tellers
ID: 4
Index: tellers_pkey
12) On the master node, create a new subscription for each slave using the following options:
Origin: 1
Provider: 1 - Master node
Receiver: 10 - Slave node 1

Origin: 1
Provider: 1 - Master node
Receiver: 20 - Slave node 2
13) Start the slon service (or daemons on *nix):
> net start Slony-I
14) Initial replication should begin and can be monitored on the statistics tab in pgAdmin for each node. The pgbench utility may be run against the master database to generate a test workload.

Monday, 6 August 2007

OpenSSL on Windows

OK, to get the usual intro over and done with - blog, neglected, back now, very sorry, won't do it again.

And now onto business... Ben West reported a problem with pgAdmin III v.1.8.0 Beta 2 on Windows which seems to have gained a dependency on MSVCR71.DLL (that's the Visual C++ runtime library for VC++ 7.1). pgAdmin is built with VC++ 8 these days so we really don't want to ship the 7.1 runtimes. After a little investigation, it seems that the latest builds of OpenSSL from Shining Light Productions are built using VC++ 7.1 :-(

So, it seems we need to build OpenSSL ourselves now to avoid this issue. I struggled to find the details of how to do that, so after finding the relevant info elsewhere, I'll repeat it here for future reference...

So, here's how it's done. You will need something to unpack the OpenSSL source code (such as tar with gzip support), and a perl installation, as well as VC++.

  1. Download the OpenSSL Source.

  2. Start a Visual Studio command prompt for the version of Visual C++ you need to compile against (8.0 for us, otherwise known as 2005).

  3. Unpack the OpenSSL source, and CD into the source directory.

  4. Configure the source code:

    C:\openssl-0.9.8e> perl configure VC-WIN32

  5. Generate assembler code for speed:

    C:\openssl-0.9.8e> ms\do_masm

  6. Finally, the libraries can be built:

    C:\openssl-0.9.8e> nmake -f ms\ntdll.mak

And that's it! The OpenSSL DLLs should be found in the out32dll directory.