By ghempton


2009-09-08 22:13:45 8 Comments

A common scenario when I develop is that the codebase will have several config files which require machine specific settings. These files will be checked into Git and other developers will always accidentally check them back in and break someone else's configuration.

A simple solution to this would be to just not check them in to Git, or even to additionally add a .gitignore entry for them. However, I find that it is much more elegant to have some sensible defaults in the file which the developer can modify to suit his needs.

Is there an elegant way to make Git play nicely with such files? I would like to be able to modify a machine-specific configuration file and then be able to run "git commit -a" without checking that file in.

10 comments

@yvess 2019-02-12 08:29:38

Nowadays (2019) I use ENV vars for example in python/django, you can also add defaults to them. In the context of docker I can save the ENV vars in a docker-compose.yml file or an extra file which is ignored in version control.

# settings.py
import os
DEBUG = os.getenv('DJANGO_DEBUG') == 'True'
EMAIL_HOST = os.environ.get('DJANGO_EMAIL_HOST', 'localhost')

@Danilo Souza Morães 2018-07-22 18:10:52

Building on @Greg Hewgill's answer, you could add a specific commit with your local changes and tag it as localchange:

git checkout -b feature master
vim config.local
git add -A && git commit -m "local commit" && git tag localchange

Then proceed to add your feature's commits. After finishing the work, you can merge this branch back to master without the localchange commit by doing this:

git rebase --onto master localchange feature
git fetch . feature:master
git cherry-pick localchange
git tag localchange -f

These commands will:

1) Rebase your feature branch to master, ignoring the localchange commit. 2) Fast forward master without leaving feature branch 3) Add localchange commit back to the top of the feature branch so you can continue working on it. You can do this to any other branch you want to continue working on. 4) Reset localchange tag to this cherry-picked commit so we can use rebase --onto again in the same way.

This isn't meant to replace the accepted answer as the best general solution, but as a way of thinking out of the box about the problem. You basically avoid accidentally merging local changes to master by only rebasing from localchange to feature and fast forwarding master.

@Pianoman 2015-01-17 20:05:47

I agree with the best answer but also would like add something. I use an ANT script to strip & modify files from the GIT repo so I'm sure no production files get overwritten. There is a nice option in ANT to modify java-property files. This means putting your local test variables in a java-style property file and adding some code to process it, but it gives you the opportunity to automate building your site before you FTP it online. Typically you would put your production information in the site.default.properties file, and let ANT manage the settings. Your local settings would be in the site.local.properties.

    <?php
/**
 * This class will read one or two files with JAVA style property files. For instance site.local.properties & site.default.properties
 * This will enable developers to make config files for their personal development environment, while maintaining a config file for 
 * the production site. 
 * Hint: use ANT to build the site and use the ANT <propertyfile> command to change some parameters while building.
 * @author martin
 *
 */
class javaPropertyFileReader {

    private $_properties;
    private $_validFile;

    /**
     * Constructor
     * @return javaPropertyFileReader
     */
    public function   __construct(){
        $this->_validFile = false;
        return $this;
    }//__construct

    /**
     * Reads one or both Java style property files
     * @param String $filenameDefaults
     * @param String $filenameLocal
     * @throws Exception
     * @return javaPropertyFileReader
     */
    public function readFile($filenameDefaults, $filenameLocal = ""){

        $this->handleFile($filenameDefaults);
        if ($filenameLocal != "") $this->handleFile($filenameLocal);
    }//readFile

    /**
     * This private function will do all the work of reading the file and  setting up the properties
     * @param String $filename
     * @throws Exception
     * @return javaPropertyFileReader
     */
    private function handleFile($filename){

    $file = @file_get_contents($filename);

    if ($file === false) {
         throw (New Exception("Cannot open property file: " . $filename, "01"));
    }
    else {
        # indicate a valid file was opened
        $this->_validFile = true;

        // if file is Windows style, remove the carriage returns
        $file = str_replace("\r", "", $file);

        // split file into array : one line for each record
        $lines = explode("\n", $file);

        // cycle lines from file
        foreach ($lines as $line){
            $line = trim($line);

            if (substr($line, 0,1) == "#" || $line == "") {
                #skip comment line
            }
            else{
                // create a property via an associative array
                $parts   = explode("=", $line);
                $varName = trim($parts[0]);
                $value   = trim($parts[1]);

                // assign property
                $this->_properties[$varName] = $value;
            }
        }// for each line in a file
    }
    return $this;
    }//readFile

    /**
     * This function will retrieve the value of a property from the property list.
     * @param String $propertyName
     * @throws Exception
     * @return NULL or value of requested property
     */
    function getProperty($propertyName){
        if (!$this->_validFile) throw (new Exception("No file opened", "03"));

        if (key_exists($propertyName, $this->_properties)){
            return $this->_properties[$propertyName];
        }
        else{
          return NULL;
        }
    }//getProperty

    /**
     * This function will retreive an array of properties beginning with a certain prefix.
     * @param String $propertyPrefix
     * @param Boolean $caseSensitive
     * @throws Exception
     * @return Array
     */
    function getPropertyArray($propertyPrefix, $caseSensitive = true){
        if (!$this->_validFile) throw (new Exception("No file opened", "03"));

        $res = array();

        if (! $caseSensitive) $propertyPrefix= strtolower($propertyPrefix);

        foreach ($this->_properties as $key => $prop){
            $l = strlen($propertyPrefix);

            if (! $caseSensitive) $key = strtolower($key);

            if (substr($key, 0, $l ) == $propertyPrefix) $res[$key] = $prop;
        }//for each proprty

        return $res;
    }//getPropertyArray

    function createDefineFromProperty($propertyName){
        $propValue = $this->getProperty($propertyName);
        define($propertyName, $propValue);
    }//createDefineFromProperty


    /**
     * This will create a number of 'constants' (DEFINE) from an array of properties that have a certain prefix.
     * An exception is thrown if 
     * @param  String $propertyPrefix
     * @throws Exception
     * @return Array The array of found properties is returned.
     */
    function createDefinesFromProperties($propertyPrefix){
        // find properties
        $props = $this->getPropertyArray($propertyPrefix);

        // cycle all properties 
        foreach($props as $key => $prop){

            // check for a valid define name
            if (preg_match("'[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*'", $key)) {
                define($key, $prop);
            }   
            else{
                throw (new Exception("Invalid entry in property file: cannot create define for {" . $key . "}", "04"));
            }   
        }// for each property found

        return $props;
    }//createDefineFromProperty

}//class javaPropertyFileReader

then use it:

  $props = new javaPropertyFileReader();
  $props->readFile($_SERVER["DOCUMENT_ROOT"] . "/lib/site.default.properties",$_SERVER["DOCUMENT_ROOT"] . "/lib/site.local.properties");

  #create one DEFINE
  $props->createDefineFromProperty("picture-path");

  # create a number of DEFINEs for enabled modules
  $modules = $props->createDefinesFromProperties("mod_enabled_");

Your site.default.properties would look like:

release-date=x
environment=PROD
picture-path=/images/

SITE_VERSION_PRODUCTION=PROD
SITE_VERSION_TEST=TEST
SITE_VERSION_DEVELOP=DEV

# Available Modules
mod_enabled_x=false
mod_enabled_y=true
mod_enabled_z=true

and your site.local.properties would look like (notice the difference environment and enabled modules):

release-date=x
environment=TEST
picture-path=/images/

SITE_VERSION_PRODUCTION=PROD
SITE_VERSION_TEST=TEST
SITE_VERSION_DEVELOP=DEV

# Available Modules
mod_enabled_x=true
mod_enabled_y=true
mod_enabled_z=true

And your ANT instructions: ($d{deploy} being your deployment target directory)

<propertyfile
    file="${deploy}/lib/site.properties"
    comment="Site properties">
    <entry  key="environment" value="PROD"/>
    <entry  key="release-date" type="date" value="now" pattern="yyyyMMddHHmm"/>
</propertyfile>

@yvess 2011-12-08 20:48:19

I do it like it's recommended here with default and local config files. To manage my local config files wich are in the projects .gitignore, I made a git repo ~/settings. There I manage all my local settings from all projects. You create, for example a folder project1 in ~/settings and put all the local config stuff for this project into it. After that you can symlink that files/folder to your project1.

With that approach you can track your local config files, and don't put them into to the normal source code repository.

@bhuber 2011-11-30 14:36:10

You can try git update-index --skip-worktree filename . This will tell git to pretend that local changes to filename don't exist, so git commit -a will ignore it. It has the added advantage of also resisting git reset --hard, so you won't accidentally lose your local changes. Also, automatic merges will fail gracefully if the file is changed upstream (unless the working directory copy matches the index copy, in which case it will be automatically updated). The downside is the command has to be run on all machines involved, and it's difficult to do this automatically. See also git update-index --assume-unchanged for a subtly different version of this idea. Details on both can be found with git help update-index .

@Senseful 2014-08-15 00:13:49

You can find more information about these commands in the question Difference Between 'assume-unchanged' and 'skip-worktree'. From the top answer, it looks like you want --skip-worktree in this case.

@crazy2be 2011-05-01 19:03:47

The simplest solution is to edit the file to defaults, commit it, then add it to your .gitignore. This way, developers will not accidentally commit it when doing git commit -a, but they can still commit it in the (presumably rare) case where you want to change your defaults with git add --force.

However, having a .default and .local config file is ultimately the best solution, since this allows anyone with a machine-specific configuration to change the defaults, without having to break their own configuration.

@Zeemee 2019-01-21 14:14:03

This doesn't work - if the files are tracked and added to the .gitignore later, changes are still tracked.

@Phil Miller 2009-09-09 01:40:37

Have your program read a pair of configuration files for its settings. First, it should read a config.defaults file that would be included in the repository. Then, it should read a config.local file that should be listed in .gitignore

With this arrangement, new settings appear in the defaults file and take effect as soon as it's updated. They will only vary on particular systems if they're overridden.

As a variation on this, you could have just a general config file that you ship in version control, and have it do something like include config.local to bring in the machine-specific values. This introduces a more general mechanism (versus policy) in you code, and consequently enables more complicated configurations (if that's desirable for your application). The popular extension from this, seen in many large-scale open-source software, is to include conf.d, which reads configuration from all the files in a directory.

Also see my answer to a similar question.

@ghempton 2009-09-12 16:39:57

I'm gonna give this the answer. This method achieves the desired effect, with the only downside that it requires extra logic on the part of the application.

@Bryan Alves 2009-09-09 14:06:25

Check in a default configuration with a different extension (say .default), use a symlink to symlink the default to the correct location, add the correct location to .gitignore, and add everything else related to the configuration to .gitignore (so the only thing that gets checked in is config.default).

Additionally, write a quick install script that sets up the symlinks for your application-wide.

We used a similar approach at a previous company. The install script autodetected what environment you were running in (sandbox, development, QA, production), and would automatically do the right thing. If you had a config.sandbox file, and were running from the sandbox, it would link that (otherwise it would just link the .defaults file). Common procedure was to copy .defaults and change settings as necessary.

Writing the install script is easier than you might imagine, and gives you a lot of flexibility.

@Greg Hewgill 2009-09-08 23:31:52

Another approach is to maintain local changes to common configuration files in another private branch. I do this for some projects that require several local changes. This technique may not be applicable to all situations, but it works for me in some cases.

First I create a new branch based on the master branch (in this particular case I'm using git-svn so I need to commit from master but that's not terribly important here):

git checkout -b work master

Now modify the configuration file(s) as necessary and commit. I usually put something distinctive in the commit message like "NOCOMMIT" or "PRIVATE" (this will be useful later). At this point, you can work away on your private branch using your own config file.

When you want to push your work back upstream, cherry-pick each change from your work branch to the master. I have a script to help do this, which looks something like this:

#!/bin/sh

BRANCH=`git branch | grep ^\\* | cut -d' ' -f2`
if [ $BRANCH != "master" ]; then
  echo "$0: Current branch is not master"
  exit 1
fi

git log --pretty=oneline work...master | grep -v NOCOMMIT: | cut -d' ' -f1 | tac | xargs -l git cherry-pick

This first checks to make sure I'm on the master branch (sanity check). Then, it lists each commit in work, filters out the ones that mention the NOCOMMIT keyword, reverses the order, and finally cherry-picks each commit (now from the oldest first) into master.

Finally, after pushing the changes in master upstream, I switch back to work and rebase:

git checkout work
git rebase master

Git will reapply each of the commits in the work branch, effectively skipping over the one(s) that have already been applied in master through the cherry-picking. What you should be left with is only the NOCOMMIT local commits.

This technique makes the push process a bit more time-consuming, but it solved a problem for me so I thought I'd share.

@Phil Miller 2009-09-09 01:50:58

You realize you're asking the oblivious not-question-asker to do this? The person who just runs git commit -a without a care in the world?

@Danilo Souza Morães 2018-07-22 17:55:04

Following that same strategy, you could tag the commit where you set your local config files and use a combination of git rebase --onto and git fetch to do the same thing

@hgmnz 2009-09-08 22:19:58

One possibility is to have the actual files in your .gitignore, but check in default configurations with a different extension. A typical example for a Rails app would be the config/database.yml file. We would check in config/database.yml.sample, and each developer creates their own config/database.yml which is already .gitignored.

@ghempton 2009-09-08 22:43:31

Yeah this is an incremental improvement, but it is still not optimal since if the checked in version is intentionally changed, it is not reflected on the developer config files. An example of when that would be useful is when a new property is added etc.

@Brian Kelly 2009-09-09 01:58:55

That could be address with good commit notes and descriptive error messages that complain when a property isn't set. Also an email communicating to your team about the change helps.

@Senseful 2014-09-03 19:34:09

For more information about this solution and a great example, see this answer.

Related Questions

Sponsored Content

35 Answered Questions

[SOLVED] How to remove local (untracked) files from the current Git working tree

  • 2008-09-14 09:06:10
  • Readonly
  • 2136385 View
  • 6537 Score
  • 35 Answer
  • Tags:   git branch git-branch

81 Answered Questions

[SOLVED] How do I undo the most recent local commits in Git?

13 Answered Questions

[SOLVED] Move the most recent commit(s) to a new branch with Git

42 Answered Questions

[SOLVED] How do I revert a Git repository to a previous commit?

33 Answered Questions

[SOLVED] How do I undo 'git add' before commit?

27 Answered Questions

[SOLVED] How to modify existing, unpushed commit messages?

42 Answered Questions

[SOLVED] How do I force "git pull" to overwrite local files?

32 Answered Questions

[SOLVED] How can I reset or revert a file to a specific revision?

23 Answered Questions

[SOLVED] Commit only part of a file in Git

  • 2009-07-06 02:25:02
  • freddiefujiwara
  • 357447 View
  • 2615 Score
  • 23 Answer
  • Tags:   git git-commit

21 Answered Questions

[SOLVED] Ignore files that have already been committed to a Git repository

Sponsored Content