4.11.10. HOWTO Work with the SIMP Rsync Shares¶
When we added support for multiple environments, the SIMP rsync space in
/var/simp/environments/<environment>/rsync
became quite complex.
This will guide you through the new rsync layout as well as providing guidance on setting up new rsync shares for your various components.
This is very SIMP-specific and does not preclude you from using rsync however you like. However, if you want multi-environment support, you will need to replicate something like what we have done for your custom directories.
4.11.10.1. Why SIMP Uses Rsync¶
Rsync support was introduced in SIMP in the early days due to the fact that the Puppet native file synchronization mechanisms were relatively horrible at syncing large files (too much in memory) and large numbers of files in a directory tree (too many resources and system load).
Rsync neatly solves both of these issues and is present on all SIMP systems by default.
By default, SIMP wraps all rsync
connections in a Stunnel
connection to provide encrypted connections. Additionally, SIMP adds randomly
generated passwords to sensitive shares to prevent unauthorized connections.
You can restrict this as far as necessary in your environment but the defaults should suit most needs.
4.11.10.2. Standard Capabilities¶
The standard SIMP rsync shares for the production
Puppet environment
exist at /var/simp/environments/production/rsync
. This is an assumed path
and changing this path will break some aspects of the system.
Within this directory, you will find a set of files with the name .shares
.
This file is used by the fact simp_rsync_environments
to indicate that all
directories at the given location should be added as rsync shared directories.
The data structure in the simp_rsync_environments
is based on the lower
cased name of the containing directory. This means that there should
NEVER be two directories with the same name at the same level of the
directory hierarchy. In this case, the last one present alphabetically will
win.
As a concrete example, given the following directory structure:
var
└── simp
└── environments
└── production
└── rsync
├── Global
│ ├── .shares
│ └── clamav
│ ├── main.cvd
│ ├── daily.cld
│ └── bytecode.cld
├── .rsync.facl
├── README
└── RedHat
├── Global
│ ├── freeradius
│ ├── .shares
│ ├── tftpboot
│ │ └── linux-install
│ │ └── README
│ ├── dhcpd
│ │ ├── dhcpd.conf
│ │ └── LICENSE
│ ├── snmp
│ │ ├── mibs
│ │ └── dlmod
│ └── apache
│ └── www
│ ├── cgi-bin
│ ├── error
│ │ └── include
│ ├── icons
│ │ └── small
│ └── html
├── 6
│ ├── bind_dns
│ │ └── LICENSE
│ └── .shares
└── 7
├── bind_dns
│ └── LICENSE
└── .shares
The following would be returned by the simp_rsync_environments
fact:
{
"production": {
"id": "production",
"rsync": {
"id": "rsync",
"global": {
"id": "Global",
"shares": [
"clamav"
]
},
"redhat": {
"id": "RedHat",
"6": {
"id": "6",
"shares": [
"bind_dns"
]
},
"7": {
"id": "7",
"shares": [
"bind_dns"
]
},
"global": {
"id": "Global",
"shares": [
"freeradius",
"tftpboot",
"dhcpd",
"snmp",
"apache"
]
}
}
}
}
}
Breaking this down, the following data is shown:
{
"downcased_directory_name": {
"id": "Original_Directory_Name",
"downcased_subdirectory_name": {
"id": "Original_Subdirectory_Name",
"shares": [
"Directory One",
"directory two"
]
}
}
}
Note
The presence of the .shares
file in the directory tree tells the
simp_rsync_environments
fact that all directories at that level are to
be exposed as shares in the returned data structure.
That said, it is up to your Puppet logic to actually expose them as such!
See the simp::server::rsync_shares
class to see how we do this for the
default rsync shares.
4.11.10.3. Supporting Additional Environments¶
Generally, in a SIMP environment, you are going to want to start with the directory structure that we have and simply copy the entire data structure to a directory with your custom name.
Warning
Be sure not to copy any sensitive information into the space!
For example, if you wanted to create the standard dev/test/prod structure, and
the production
environment already existed:
cd /var/simp/environments
cp -a production dev
cp -a production test
Alternatively, you can use the simp environment new
command to affect the
copy of all or some the SIMP Omni-Environment. (You can also use that
command to affect links of the SIMP Secondary Environment or
SIMP Writable Environment, which in some circumstances may be more
appropriate.)
After this, you will now have an enhanced simp_rsync_environments
data
structure that holds all of the information for the dev
, test
, and
production
environments.
You can then manipulate the contents of the different environments to suit your needs.
Note
The contents of the various rsync directories are not under version control by default. While you may add them to a VCS of your choosing (SVN, Git, etc…), there may be some VERY large files present in these directories.
Make sure your system can handle the load before adding rsync content into a VCS!
4.11.10.4. Disabling Stunnel¶
If you decide to disable stunnel, you will need to specify your rsync server in Hiera, if it is not already specified.
Warning
If you disable stunnel, your data and any rsync access credentials will be passed in the clear!
---
simp_options::rsync::server: <rsync_server_fqdn>
Additionally, you will need to ensure your firewall is open on the rsync port. Include the following on the node acting as the rsync server.
class site::rsync_iptables (
Simplib::Netlist $allow = simplib::lookup('simp_options::trusted_nets'),
Simplib::Port $rsync_port = 873
){
iptables::listen::tcp_stateful { "rsync_shares":
trusted_nets => $allow,
dports => $rsync_port
}
}