14 Replies - 1259 Views - Last Post: 16 June 2018 - 11:04 AM

#1 kathy025   User is offline

  • D.I.C Head

Reputation: 41
  • View blog
  • Posts: 190
  • Joined: 19-December 14

Support for private branches in Git repos

Posted 10 June 2018 - 05:09 PM

We usually have either a public or a private Git repo but is it technically possible to have a public repo with private branches or visibility control?

Rationale:
  • To backup WIP branches to remote repo.
  • To allow only specific devs access to a branch (visibility control).
  • To hide sensitive (e.g. company) data but keep the rest of the code open-source. For example, a test framework is free for all but when you branch off and start putting company-sensitive test data, it now needs protection but only want company colleagues to have access to the branch.


Is it not possible to do? Or are there workarounds (other than maintaining a parallel private repo)?
Would be great to hear some thoughts. Thanks!

Is This A Good Question/Topic? 0
  • +

Replies To: Support for private branches in Git repos

#2 Skydiver   User is online

  • Code herder
  • member icon

Reputation: 6661
  • View blog
  • Posts: 22,741
  • Joined: 05-May 12

Re: Support for private branches in Git repos

Posted 10 June 2018 - 05:34 PM

As much as there is much derision (and warnings) about using the git submodules feature, this maybe as appropriate use of the feature. Keep the submodules as private repos with limited access, while the root repo remains public.
Was This Post Helpful? 1
  • +
  • -

#3 jon.kiparsky   User is online

  • Beginner
  • member icon


Reputation: 11374
  • View blog
  • Posts: 19,406
  • Joined: 19-March 11

Re: Support for private branches in Git repos

Posted 10 June 2018 - 05:37 PM

I do not know of any way to do what you're trying to do. The rationales you cite do not seem to align with generally accepted practice - for example, I wouldn't really want to work on a team that was hiding their WIP (even though I'm not really in the habit of monitoring my teammates' work before they issue a PR)

Quote

To hide sensitive (e.g. company) data but keep the rest of the code open-source.


In general, separation of code and data should take care of this. Don't commit sensitive information to a repo, period.

Quote

For example, a test framework is free for all but when you branch off and start putting company-sensitive test data


Why on earth would you ever do this?
Was This Post Helpful? 2
  • +
  • -

#4 kathy025   User is offline

  • D.I.C Head

Reputation: 41
  • View blog
  • Posts: 190
  • Joined: 19-December 14

Re: Support for private branches in Git repos

Posted 10 June 2018 - 06:04 PM

View PostSkydiver, on 11 June 2018 - 08:34 AM, said:

As much as there is much derision (and warnings) about using the git submodules feature, this maybe as appropriate use of the feature. Keep the submodules as private repos with limited access, while the root repo remains public.

I was not aware there is such a feature of submodules. Thank you for mentioning.

View Postjon.kiparsky, on 11 June 2018 - 08:37 AM, said:

I wouldn't really want to work on a team that was hiding their WIP (even though I'm not really in the habit of monitoring my teammates' work before they issue a PR)

The intention is not to hide the WIP branch, but rather have a handy copy in the remote should something unprecedented happens to the local - akin to uploading files to a private GDrive just in case.

View Postjon.kiparsky, on 11 June 2018 - 08:37 AM, said:

In general, separation of code and data should take care of this. Don't commit sensitive information to a repo, period.

Why on earth would you ever do this?

While not committing sensitive data is ideal, reality check is that it is not improbable (and sometimes even considered). Even if you separate data, config files (e.g. *.properties) and test data (e.g. excel, csv) still fly off some other (unsafe) method anyway like emails. I had a project manager email username/password for like 50 people in CC.

A test framework can be generically used for any application. When people decide to take the framework and use it in their own applications, that is where application/company-specific test data comes in. It can take a different PoV compared to developing web apps or mobile apps.

This post has been edited by kathy025: 10 June 2018 - 06:06 PM

Was This Post Helpful? 0
  • +
  • -

#5 Skydiver   User is online

  • Code herder
  • member icon

Reputation: 6661
  • View blog
  • Posts: 22,741
  • Joined: 05-May 12

Re: Support for private branches in Git repos

Posted 10 June 2018 - 06:31 PM

View Postkathy025, on 10 June 2018 - 09:04 PM, said:

While not committing sensitive data is ideal, reality check is that it is not improbable (and sometimes even considered). Even if you separate data, config files (e.g. *.properties) and test data (e.g. excel, csv) still fly off some other (unsafe) method anyway like emails. I had a project manager email username/password for like 50 people in CC.

Client side and server side git hooks may help prevent stuff like this. The server side hooks can reject attempts to commit stuff that should not be committed, even in a WIP branch of a public repo.

The client side hooks will require discipline on the part of your users that they enable them each time after they clone. IF your team uses JIRA and uses its "create a branch for this issue feature", and if your team members are lazy and click on the JIRA+SourceTree option to clone the repo and make that branch current, THEN client side hooks may not be for your team since your team members are more likely than not to enable the client side hooks.
Was This Post Helpful? 1
  • +
  • -

#6 kathy025   User is offline

  • D.I.C Head

Reputation: 41
  • View blog
  • Posts: 190
  • Joined: 19-December 14

Re: Support for private branches in Git repos

Posted 10 June 2018 - 06:47 PM

@Sky:
I even have devs sending me db credentials via Skype so was looking for some controlled medium over nothing. I see. There can be some complexity involved but hooks + agreed practice (discipline) can work.
Was This Post Helpful? 0
  • +
  • -

#7 Skydiver   User is online

  • Code herder
  • member icon

Reputation: 6661
  • View blog
  • Posts: 22,741
  • Joined: 05-May 12

Re: Support for private branches in Git repos

Posted 10 June 2018 - 06:55 PM

View Postkathy025, on 10 June 2018 - 09:04 PM, said:

A test framework can be generically used for any application. When people decide to take the framework and use it in their own applications, that is where application/company-specific test data comes in. It can take a different PoV compared to developing web apps or mobile apps.

There is something dramatically wrong with a test framework of you need to modify the source code of the framework to be able to run tests. I suspect that 85% if the time you are using the framework the wrong way if you find yourself in that situation. The other 10% of the time, there is a bug for which you have a fix. You should really be contributing that fix back to the public domain, as well sharing with your users. The last 5% is that you are using the wrong framework for the job and so you have some proprietary kludge to try to fit a square peg into a round hole. You should still share that fork for the sake of continuity of business.

Anyway, looking at that link you have, it seems that it is meant to be a template for all the boilerplate that you would need to use Cucumber. It is not the Cucumber test framework itself. The intent of that template is for it to be readonly. Essentially, people are supposed to copy that tree, hack on it, and then commit into THEIR OWN repo. Or in git terms, people are supposed to fork that repo, and then never push back to the upstream repo. So what you can do is put a server side hook on that template repo that rejects all pushes.
Was This Post Helpful? 0
  • +
  • -

#8 Skydiver   User is online

  • Code herder
  • member icon

Reputation: 6661
  • View blog
  • Posts: 22,741
  • Joined: 05-May 12

Re: Support for private branches in Git repos

Posted 10 June 2018 - 07:09 PM

View Postkathy025, on 10 June 2018 - 09:04 PM, said:

The intention is not to hide the WIP branch, but rather have a handy copy in the remote should something unprecedented happens to the local - akin to uploading files to a private GDrive just in case.

git does not limit you to a single remote. Your local repo can have many remotes. For stuff that I find interesting enough to work on even during my time off and does not contain any proprietary information, I often setup an extra remote that points to a private repo I setup externally. Each time I push to our internal git server, I will also post to my private repo. At home I clone and work against that private repo. When I get back to work, I pull from my private repo, and then pick up where I left off at home.

Anyway, your team members can do the same. They can have other repos and add them as remotes. If your team is using Enterprise Bitbucket (formerly known as Stash), then each developer automatically gets private repos on top of the team/project repos. Just go to your profile, and you can create more private repos.
Was This Post Helpful? 1
  • +
  • -

#9 kathy025   User is offline

  • D.I.C Head

Reputation: 41
  • View blog
  • Posts: 190
  • Joined: 19-December 14

Re: Support for private branches in Git repos

Posted 10 June 2018 - 07:31 PM

@Sky:
It appears a parallel private repo is still a pragmatic solution (which was what I was considering earlier). Thank you Sky for the many tips.
Was This Post Helpful? 0
  • +
  • -

#10 jon.kiparsky   User is online

  • Beginner
  • member icon


Reputation: 11374
  • View blog
  • Posts: 19,406
  • Joined: 19-March 11

Re: Support for private branches in Git repos

Posted 10 June 2018 - 08:00 PM

I just want to point out that "pragmatic" is not a synonym for "bad, but we're doing it anyway". If your organization does not have anyone with experience managing secrets, hire a contractor and ask them to help you get set up with realistic procedures and policies.
That being done, I'd suggest you push for resources to get someone on your team (possibly you, since you seem to at least be thinking about these matters) to get schooled up on dev ops so that you'll have that talent in-house when questions like this arise in future.
Was This Post Helpful? 1
  • +
  • -

#11 kathy025   User is offline

  • D.I.C Head

Reputation: 41
  • View blog
  • Posts: 190
  • Joined: 19-December 14

Re: Support for private branches in Git repos

Posted 10 June 2018 - 11:03 PM

@Jon: I don't take "pragmatic" as a bad word either.

prag·mat·ic
praɡˈmadik/
adjective
dealing with things sensibly and realistically in a way that is based on practical rather than theoretical considerations.

I take the dictionary definition, but yes, the term has gotten some bad rep in the IT industry.
Was This Post Helpful? 0
  • +
  • -

#12 ndc85430   User is online

  • I think you'll find it's "Dr"
  • member icon

Reputation: 934
  • View blog
  • Posts: 3,740
  • Joined: 13-June 14

Re: Support for private branches in Git repos

Posted 11 June 2018 - 10:46 AM

View Postkathy025, on 11 June 2018 - 01:09 AM, said:

To backup WIP branches to remote repo.


This sounds odd to me. I see from your later post in this thread that you just want to do backups. You are aware, though, that each clone of a repo has the whole history, right? So, you already have backups because multiple people are likely to have the repo cloned on their machines. Of course, Skydiver's point about multiple remotes is also useful.

Having said that, I do wonder about how your team operates. Do you find you have integration hell when you have long-lived branches that people work on separately and you try to bring everything together only at the end? If so, you might want to consider trunk based development - essentially, working only on master with no branches at all. This does require the team to be communicating a lot, as well as working in small, focussed commits (why wouldn't you do that anyway?) and using things like feature toggles when a particular piece of work shouldn't be enabled in production (because it's work in progress, but still deployable). A good blog post about this kind of thing is titled "As soon as you can integration". At my last place of work, integration hell was so common. We work with no branches at all at my current place and really, I can't say I miss them!

This post has been edited by ndc85430: 15 June 2018 - 09:31 AM

Was This Post Helpful? 1
  • +
  • -

#13 jon.kiparsky   User is online

  • Beginner
  • member icon


Reputation: 11374
  • View blog
  • Posts: 19,406
  • Joined: 19-March 11

Re: Support for private branches in Git repos

Posted 11 June 2018 - 11:38 AM

View Postkathy025, on 11 June 2018 - 01:03 AM, said:

dealing with things sensibly and realistically in a way that is based on practical rather than theoretical considerations.


Fair enough. Take it from me, pragmatic does not include ignoring best-practice advice from people who have already made the mistakes and learned from them. That would be neither sensible nor realistic, and it would ignore some very practical considerations.

I recognize that you're pushing back because you want to put this issue to bed so you can get on with shipping code, and that's respectable. All I'm saying is, there's good reasons not to want the things you seem to want to want. Maybe it would be good to step back and say a little more about what you're actually trying to do?

Of you can just get on with it if you like - but it's going to be painful.

View Postndc85430, on 11 June 2018 - 12:46 PM, said:

You are aware, though, that each clone of a repo has the whole history, right?


Solid point. Also makes it really clear why secrets don't belong in repositories - nothing ever leaves the repo, short of a catastrophic server oopsie.
Was This Post Helpful? 0
  • +
  • -

#14 Skydiver   User is online

  • Code herder
  • member icon

Reputation: 6661
  • View blog
  • Posts: 22,741
  • Joined: 05-May 12

Re: Support for private branches in Git repos

Posted 11 June 2018 - 12:12 PM

View Postjon.kiparsky, on 11 June 2018 - 02:38 PM, said:

Solid point. Also makes it really clear why secrets don't belong in repositories - nothing ever leaves the repo, short of a catastrophic server oopsie.

Or a misconfiguration like giving force push rights to the wrong people... :)
Was This Post Helpful? 0
  • +
  • -

#15 ndc85430   User is online

  • I think you'll find it's "Dr"
  • member icon

Reputation: 934
  • View blog
  • Posts: 3,740
  • Joined: 13-June 14

Re: Support for private branches in Git repos

Posted 16 June 2018 - 11:04 AM

Also, for storing credentials, etc. there are tools like Vault, which is what we're using.
Was This Post Helpful? 0
  • +
  • -

Page 1 of 1