10 Replies - 2140 Views - Last Post: 02 January 2014 - 10:45 PM

#1 Nykc  Icon User is offline

  • Gentleman of Leisure
  • member icon

Reputation: 731
  • View blog
  • Posts: 8,644
  • Joined: 14-September 07

Github - best practices relating to project/repo management

Posted 26 November 2013 - 10:37 AM

Working with Github on a new project and I have a few questions regarding the appropriate way to do this. My project is PHP based and I am using a few dependencies along with Composer to manage them. No big deal and rather common. So I created an initial commit, however included everything - dependencies and all. I got to thinking this is probably not the best way to package my project. After installing composer and markdown on my machine, all I have to do is run a composer update and the dependencies will be downloaded and installed, also I have the issue of version management, if one of my dependencies is updated I too would have to update. Should I be omitting all these from my Github repository and make a note in the README.md file saying add any dependencies in config.json or just install composer and run a composer install.


So I have my config.json, config.lock and .htaccess file which I currently only have .htaccess in my gitignore file, should I also add the config.lock to the gitignore?

Trivial things, but I am curious as to best practices and how to efficiently manage the repo without constantly updating the dependencies or bogging down my repo with them when they can be installed separately. I eventually would like to package and have it installed similar to a wordpress etc..

Thanks

Is This A Good Question/Topic? 0
  • +

Replies To: Github - best practices relating to project/repo management

#2 no2pencil  Icon User is offline

  • Admiral Fancy Pants
  • member icon

Reputation: 5364
  • View blog
  • Posts: 27,325
  • Joined: 10-May 07

Re: Github - best practices relating to project/repo management

Posted 26 November 2013 - 10:41 AM

My opinion in the best practice would depend on how you want the project referenced. For some projects, I just want a code repository, therefor the kitchen sink doesn't need to be included. Other projects, however, I have a procedure to install the tarball, drag & drop, as part of the installation. Thus the kitchen sink is expected to be included in the tarball package.
Was This Post Helpful? 1
  • +
  • -

#3 Nykc  Icon User is offline

  • Gentleman of Leisure
  • member icon

Reputation: 731
  • View blog
  • Posts: 8,644
  • Joined: 14-September 07

Re: Github - best practices relating to project/repo management

Posted 26 November 2013 - 10:52 AM

I was thinking of just writing a shell script to handle getting the dependencies, maybe even installing composer in the project for them as well. I just feel like putting my dependencies in the repo is not the best way to handle this. I am aiming for lightweight here, hence the reason for the project lol

Thanks for the reply
Was This Post Helpful? 0
  • +
  • -

#4 no2pencil  Icon User is offline

  • Admiral Fancy Pants
  • member icon

Reputation: 5364
  • View blog
  • Posts: 27,325
  • Joined: 10-May 07

Re: Github - best practices relating to project/repo management

Posted 26 November 2013 - 10:59 AM

Your script could create them before or after the tarball download/extraction of the git hub project. If need-be, you could also prompt them for any information & dynamically create said files.
Was This Post Helpful? 0
  • +
  • -

#5 cfoley  Icon User is online

  • Cabbage
  • member icon

Reputation: 2045
  • View blog
  • Posts: 4,233
  • Joined: 11-December 07

Re: Github - best practices relating to project/repo management

Posted 27 November 2013 - 05:47 AM

I don't know the answer and you may not get consensus from those who think they do. There are obvious advantages to both approaches.

Why don't you use branches to your favour:

Development branch: This contains only your source code. Dependencies are added to your .gitignore so you can't add them accidentally. You can create feature branches and experimental branches as normal.

Dependency branch: Only add your dependencies to this branch. There is syntax for adding files that your .gitignore blocks so use that. Make sure you check your git status carefully before committing because it will be very easy to add source files by accident.

Release branch: When your development branch reaches what could be a stable version, merge it into the release branch and also merge your dependency branch into the release branch. Use this branch to update version numbers and other release-specific issues.

Main Branch: When everything is correct in the release branch, merge it into main. Main will contain code and dependencies. However, each commit should represent a usable product with the correct configuration.

I think this represents the best of both worlds. You won't have to bother much with the dependency branch at all. If you never update your dependencies, you will only ever have to merge it into the release branch once. If you do update your dependencies, you will have a history so you can revert to earlier versions of you have to. The development branch will be purely for source code. It will have the most activity and the most sub branches. If you are strict about feature branches for new features, the head of the development branch should always represent stable code. Release it where everything is tied together and main is for finished products.

If you are uploading to github you can upload all your branches or you can just upload your development branch if you don't want to share your dependencies. With git, it's easy to call it "dev" in your local repo but "main" in your remote. There is also nothing to stop you having two repos in github: one with dependencies, the other without.

The biggest problem I can see is in applying hotfixes to main. Obviously, you want to merge them back into dev but without dragging dependencies with it. I think git allows partial merges. Hotfixes should be few and far between. Merging tham back into dev might be painful but at least it won't be too frequent.


--------------------


Another simpler option might be to have separate source trees for your code and dependencies.
Was This Post Helpful? 3
  • +
  • -

#6 Skydiver  Icon User is offline

  • Code herder
  • member icon

Reputation: 3621
  • View blog
  • Posts: 11,280
  • Joined: 05-May 12

Re: Github - best practices relating to project/repo management

Posted 27 November 2013 - 07:54 AM

To me, I guess it depends on the complexity of the project and your target "ease of use" for somebody wanting to jump in and contribute to your project. Do you expect a level of sophistication and/or good understanding of the project dependencies from any future contributors? Or do you expect any script kiddie to be able be successful on their first pull and build?

I remember being really frustrated last year when I wanted to help a DIC member help use a game engine, but it took a good half day of chasing down the dependencies and figuring out which branches NOT to get before I could even get the game engine library building.

In general, I like cfoley's recommendations above about making good use of branches. I do have some lingering questions in the back of my mind, and I'll try to express them. The coffee hasn't really kicked in yet, so hopefully they are coherent.

1. Even if there are multiple branches, if I clone a git repository won't my clone end up pulling down all the branches into my machine? Won't this repository still be huge because the dependencies branch will be in the repository, but I'm only interested in the developer branch? Or am I missing a core concept about git which let's me clone just a branch of a repository?

2. How do you get a new contributor up and running? Do you tell them to get the developer and dependencies branch? Or do you tell them to get the main branch first and then "overlay" the developer branch on top of the main branches working copy? Or some other process?
Was This Post Helpful? 1
  • +
  • -

#7 Nykc  Icon User is offline

  • Gentleman of Leisure
  • member icon

Reputation: 731
  • View blog
  • Posts: 8,644
  • Joined: 14-September 07

Re: Github - best practices relating to project/repo management

Posted 27 November 2013 - 08:30 AM

I like CFoley's gitflow approach, I was also discussing this locally with some former developers I worked with. The complexity I am looking for, I am just building a light-weight markdown/file driven blogging platform that doesn't rely on MySQL based off a tutorial I found and I want to adopt and expand upon.

It uses Composer for dependency management, so really at this point the only thing other devs would need to do, should they choose to contribute is clone the repository and run the Composer.phar or if they have Composer installed globally run a Composer Install and the dependencies would automatically be downloaded.

Currently I just plugged in everything and its a download and run essentially (there might be some .htaccess and apache2 mod_rewrite tweaks required)

One point, during my discussions that was brought to my attention was Ruby 1.8.7 vs Ruby 2.0.0 etc., some projects could break if I allow for current builds of dependencies pulled into the project. I am sure it is not going to be an issue with my project, but as with jQuery - some things get deprecated and tend to break. I am probably thinking too far ahead on this right now, this is something more of a personal project than anything, however this carries over to future projects I am working on as well that does involve a dev team and technologies where dependency management is a real issue.

Thanks for the advice guys

This post has been edited by Nykc: 27 November 2013 - 08:31 AM

Was This Post Helpful? 0
  • +
  • -

#8 cfoley  Icon User is online

  • Cabbage
  • member icon

Reputation: 2045
  • View blog
  • Posts: 4,233
  • Joined: 11-December 07

Re: Github - best practices relating to project/repo management

Posted 27 November 2013 - 10:06 AM

Another option that I think someone touched on above is to incorporate downloading and compiling dependencies in your build script. The build script would be checked into source control.

Skydiver, to address your questions:

Quote

1. Even if there are multiple branches, if I clone a git repository won't my clone end up pulling down all the branches into my machine? Won't this repository still be huge because the dependencies branch will be in the repository, but I'm only interested in the developer branch? Or am I missing a core concept about git which let's me clone just a branch of a repository?


I'm not sure. Certainly if you create a local branch, it doesn't automatically end up upstream. You have to set up a tracking branch for that. I can't remember the default cloning behaviour. You might just get main or you might get everything. I'd be surprised if there wasn't a way to have fine-grained control but that's not always convenient.

Quote

2. How do you get a new contributor up and running? Do you tell them to get the developer and dependencies branch? Or do you tell them to get the main branch first and then "overlay" the developer branch on top of the main branches working copy? Or some other process?


I don't think there is a one size fits all answer. Certainly there should be instructions available for collaborators and consideration should be given to making the process as easy as possible if encouraging participation is a goal.
Was This Post Helpful? 0
  • +
  • -

#9 Skydiver  Icon User is offline

  • Code herder
  • member icon

Reputation: 3621
  • View blog
  • Posts: 11,280
  • Joined: 05-May 12

Re: Github - best practices relating to project/repo management

Posted 27 November 2013 - 10:39 AM

View Postcfoley, on 27 November 2013 - 12:06 PM, said:

Quote

1. Even if there are multiple branches, if I clone a git repository won't my clone end up pulling down all the branches into my machine? Won't this repository still be huge because the dependencies branch will be in the repository, but I'm only interested in the developer branch? Or am I missing a core concept about git which let's me clone just a branch of a repository?


I'm not sure. Certainly if you create a local branch, it doesn't automatically end up upstream. You have to set up a tracking branch for that. I can't remember the default cloning behaviour. You might just get main or you might get everything. I'd be surprised if there wasn't a way to have fine-grained control but that's not always convenient.


Trying to educate myself while waiting for a build to complete, I've found that the correct thing to search for is "git submodules", and not "git sparse checkouts".

Anyway, as I understand things in git, when you clone a repository, you get the entire thing repo even if you are only interested in a branch of the repository. So one way around the primary repository carrying around all the dependencies is to use submodules so that the other repository carries around the dependencies, but your primary repository stays relatively lean and focused on just the source code.

I hope somebody can help clear up any misconceptions I may have picked up. I'm just doing some light skimming while monitoring a build and set of tests.
Was This Post Helpful? 1
  • +
  • -

#10 cfoley  Icon User is online

  • Cabbage
  • member icon

Reputation: 2045
  • View blog
  • Posts: 4,233
  • Joined: 11-December 07

Re: Github - best practices relating to project/repo management

Posted 27 November 2013 - 01:08 PM

I think you are right but submodules are a bit awkward to use amd come with a large set of disadvantages. I can't remember the specifics but it was bad enough to stop me using them.

Another option for this kind of thingnis subtree merging. Again, I'm not sure of the specifics but it might be something to look into.
Was This Post Helpful? 0
  • +
  • -

#11 alexdantas  Icon User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 5
  • Joined: 07-February 12

Re: Github - best practices relating to project/repo management

Posted 02 January 2014 - 10:45 PM

Just a sidenote - one of the best things you can do to your
repository is to respond to issues and pull requests on time.

I've lost count on how many good projects were seemingly
abandoned. Is very frustrating to find an awesome project,
just to see it has like 99 pending pull requests and 30
unanswered issues.

Since by default everyone gets mail notifications of their
own repositories, the least you can do is tell
"I'm busy right now" or "does somebody else have an idea"?

Authors who simply abandon their repositories are such
a bummer. Well, at least we can fork! :P/>
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1