Traceability of configuration changes using Git

When seeking a solution to keep full traceability of configuration changes made by (various) users on your firewall, the git-backup plugin might be a useful addition to your setup.

In order to use this feature, one has to install the git-backup plugin first (in System->Firmware->Plugins search for os-git-backup).

Concept

Since git backup is a little bit different than the standard backup options available, we will explain briefly how it works using the diagram below.

When config.xml changes happen due to user or api interaction, an event is triggered to which handlers can subscribe (using syshook). Our git-backup plugin subscribes to these events in order to add the received backups and commits these with information extracted from the received xml file. To prevent the system to lock during backups, we choose this loosely coupled method. Events which are yet unprocessed are being left in the (existing) backup directory.

Note

Events are processed from the moment the initial backup is configured, when disabling backups, the (local) changelog itself remains active.

On periodic intervals (the standard ones from the backup scheduler), the collected commits are pushed to the configured upstream repository. To shorten these default intervals, a custom cronjob (see Settings) can be set up, selecting Remote Backup as the Command. The regular backup procedure (which is also being triggered using the test button in the user interface) is responsible for initialising the empty local repository and configuring the upstream target.

Note

One can always change the upstream target, as long as the newly configured one is either “bare” (empty) or containing the exact same content (/change history) as the one used on this firewall.

Initial setup

The configuration part of this plugin is quite basic and offers two types of transport modes, https using a username and password combination or ssh using public key infrastructure.


Enable

Enable backup to the upstream target

URL

Target location, which defined transport protocol, options as ssh://server/project.git or https://server/project.git are allowed here.

Branch

The branch to push your commits to on the configured url

SSH private key

When using ssh, make sure to add a private key here

User Name

Username, when using gitlab and ssh, the default is git here (most of these providers use a single user and identify the user by it’s key)

password

When using https authentication, choose a password here.

Make sure to push to a “bare” upstream repository, when pressing “Setup/Test Git” the initial commits should be send to your git server.

Conflict resolution

From the user interface no conflict resolution is offered, you need to configure an upstream repository and stick to it for the lifetime of the firewall. When for some reason a backup needs to be restored and one would like to stick to the same git repository, manual conflict resolution might be an option. Support on these scenario’s is not offered.

The repository is available on the OPNsense machine in the following directory /conf/backup/git.

Note

Conflict resolution can complicate the solution a lot (merging, fast-forward, ….), for this reason we will not accept feature requests trying to push to existing (used) repositories.

Error handling

When errors occur these will be written to the normal system logging, search for git-backup in the general system logging (System -> Log Files -> General).

Some standard errors might be returned via the test button, which should provide a clear direction, known ones are:

  • authentication failure -> username/password combination is not valid or the provided ssh key doesn’t match the expected one

  • ssh hostkey changed -> it looks like a man-in-the-middle attack is happening, if that’s not the case and the remote identification changed for valid reasons, manual intervention is required (remove the offensive key from /root/.ssh/known_hosts)

  • git out of sync -> unable to synchronize, see “Conflict resolution” for additional info.

Cleanup

The repository is saved locally on the firewall in /conf/backup/git, if for some reason one would like to remove the collected history and start over from scratch, one can safetly remove this directory.

Login using a (ssh) console and remove the git directory in that case (rm -rf /conf/backup/git)

Note

As long as the plugin is installed and /conf/backup/git contains a git repository, the changes will be captured (also without an upstream). One could use this knowledge as well to keep a local (only) repository by creating a repository without assigning an upstream and leave the backup option disabled.

Tip

The firewall contains a local backup of the most recent changes (configured in System -> Configuration -> History) which the config changed event handler uses to feed to the consumers. If after a cleanup one would like to flush the collected changes again to the upstream provider, the /conf/event_config_changed.json could be removed to “forget” about the already handled config events (in which case all backups will be signaled again to all config syshook handlers)