[cfoley]

I’m looking for suggestions and hopefully a little discussion around a problem I’m facing. I’ve set up a github account and I’m shaping up a few of my projects to upload as part of a portfolio. I have about half a dozen personal projects that I have started but not brought to a sensible milestone. My plan is to shape them up and put them on github.

The problem is to do with shared code between the projects. I’m talking about little utility classes, common algorithms, reusable Swing assemblies and the like. Things that are designed as libraries from the outset (like my 2D physics engine) are always used as libraries.

Because these are personal projects, I have taken a fairly relaxed attitude to shared code over the years. At times I have had a shared project (Java project in Eclipse) which was source controlled separately. Other times, I have copied the classes into my new project and not maintained a link to any common code. Other times I have simply rewritten the classes.

As an aside, the latter has probably been the most valuable for me in terms of learning. Each time I write some code for the functionality, it is better than the last. Revisiting a problem after a period of time lets me see it in a new light.

When I am writing code professionally, a shared library is usually the way to go. However, I want to consider ease of use for anyone downloading my source (or maybe if I start something cool, forking it so they can modify and make contributions). One option using git is a separate library repository. Another is subrepositories. Both look great if you want to include a complex library but they seem like too much effort for some silly little utility classes.

Any thoughts or suggestions?

[tlhIn`toq]

cfoley, on 28 December 2012 - 08:31 AM, said:

Both look great if you want to include a complex library but they seem like too much effort for some silly little utility classes.

I notice this line of thinking with most projects, not just libraries. People building small programs don't bother separating GUI from Data for example. "Oh, its just a little app, I'll make the controls public and not bother with properties." etc.

Personally I find this leads to trouble that could have been avoided. You start getting apps (or libraries) that don't follow any rules.

Voice in my head said:

This was a small app so I did it this way.
This project was a medium app, so I did it this other way.
But this big project I followed the rules.
Oh, this started as a small project but then grew so the older code was this way but the new-ish code that way, and the latest code the other way.

If this is going to be your portfolio then impress people. Do things right the first time even if it is more work. In the long run it will be less work because everything follows the same rules, patterns and guidelines. You won't be spending 3 hours trying to remember how you did something. And you won't be confusing your contributors with Project Alpha doing one thing and Project Beta doing something else to get the same results.

[BobRodes]

Like he says.

[cfoley]

OK, so to put the question another way, does every developer worth his salt maintain a library of common code snippets that he references from all his projects?

From what little I have seen from the open source world, projects are either self-contained or they reference well-known libraries like guava or Apache Commons.

[tlhIn`toq]

I can't speak for everyone - or if anyone other than my employer thinks I'm worth my pay - But I keep a few common library files and they are used in most projects.

Sorry for blurring the names, but I want to keep my job.

• One library might be how to deal with company-specific protocols & routines like program security schemes.
  
• One might be for easing the use of the PC environment for things like multiple monitors, determining 32-64 bitness, getting free memory, getting IP addresses and user credentials.
  
• One might be for UserControls that are used a lot in order to keep a consistent look & behavior.

Thus the actual programs can be assembled by re-using code and controls that do the heavy lifting, needing only new code for purpose logic and wiring the already-created classes and controls to each other. Plus the things specific to a given application.

I'd call those libraries more than just 'snippets'. They are fairly mature namespaces.

gorf.Environment.ThisPC.Freemem returns the memory.
gorf.Environment.ThisPC.Monitors.Workspace(3) returns the workspace of the third monitor
gorf.Environment.ThisPC.IP returns the IP of the active NIC
blork.Controls.SerialPort is a complete serial port UserControl with controls and properties for COMport, baudrate etc.
wonk.RegEx.IsValidEmail returns true/false if the supplied string is a good email address.

You get the idea.

*Names and true hierarchy have been changed to protect my job.

This post has been edited by tlhIn`toq: 28 December 2012 - 10:08 AM

[tlhIn`toq]

cfoley, on 28 December 2012 - 08:31 AM, said:

Other times I have simply rewritten the classes.

As an aside, the latter has probably been the most valuable for me in terms of learning. Each time I write some code for the functionality, it is better than the last. Revisiting a problem after a period of time lets me see it in a new light.

That's the case for most of us. But how are you getting the new/better code into older projects?

If you have common library projects that are referenced in your solutions then every application gets the improved code the next time you compile. But if you are copy/pasting code all over then you have to manually scrub through all your projects-YUCK!

You want (in my opinion) to have common code appear in only one place: One file. If you find a better way to perform the SnorgleCommonLib.SortThingies() method then you update that code and increase the version number. The next time you open the WidgetProgram solution which references the existing library project of SnorgleCommonLib it will be looking at the new code without you having to do anything. It will benefit from the improvement automatically. You just open the solution and rebuild.

[BobRodes]

This last of tlhin's points out the value of interfaces. Very often you know what something is supposed to do more clearly than how to do it, so you can define a method call and continually improve the method without renaming it. This is one of the main values of interfaces and/or abstract classes; since you don't change the way the method is called, you automatically update any client calling the method with improvements.

[Skydiver]

I haven't gotten to this point yet, but having an continuous integration server may also help.

Each time you commit changes one your common libraries (you are using source control, right?), then you can cause any other projects that depend on that common library to rebuild and rerun tests. If you broke something, then you know right away. If you didn't break anything, then you automatically get updated binaries that use the latest and greatest version of your common library.

[I've been playing with Jenkins lately looking at that as a CI server to run on a AWS or Azure machine, but if anybody has any suggestions for other CI's, I'd gladly listen. Primary code written is C# targeting WPF, MVC, WinForms, and Console, and C++ targeting Win32 and Console. I've also got some PHP based websites that I'm maintaining that maybe nice to include in the things automatically built and tested.]

[AdamSpeight2008]

I use a simple NuGet Server ( a shared network folder ) for my NuGet packages for my Libraries.
If I update a one, the system report that one is available.

---

tlhIn`toq if you're being commissioned to write some software for someone.
Don't they also own any code you develop for the library whilst create the software for them?

---

Once you define a interface eg IFooBar and publicly release it, you should never change it. As it'll break existing code. Use inheritance on the inteface or define a completely new one.

This post has been edited by AdamSpeight2008: 28 December 2012 - 02:20 PM

[cfoley]

Thanks for all the useful replies everyone. TlhIn`toq, I think you have convinced me that I really do want to use libraries. But not only that but I want to use libraries for each kind of thing that I have. From memory, the major ones so far are Swing utilities and widgets, unit testing utilities, chemistry/bioinformatics classes (although the last one might be better off split further).

An integration server sounds like it might be just what I need. Definitely something to look into! Thanks for the suggestion. I have heard the term before but never really paid much attention.

My next question is how to organise all this on github. My typical setup is at least one project for my application and another for unit tests, all source controlled in one git repository. The annoying thing is Eclipse has a great mechanism for organising shared projects. For example, I could just create my Chemistry library as a set of projects in a different directory and point any of my Eclipse workspaces at it. This also makes it easy to source control seperately.

My difficulty is at the other end, once I have uploaded my separate repositories to github. It sounds like it would be a major pain for someone else (or even me if I want to set up another computer to work on a project) to download everything and link it up.

This is in stark contrast to the normal git workflow where one line of code means I'm set up and ready to go:

git clone repoistiryURL

Addendum:

It appears I'm being a big drama queen:
https://help.github....ules-with-pages

[tlhIn`toq]

AdamSpeight2008, on 28 December 2012 - 03:16 PM, said:

tlhIn`toq if you're being commissioned to write some software for someone.
Don't they also own any code you develop for the library whilst create the software for them?

I would imagine so. But I have worked for the same employer for the last 8+ years. At 50+ hours a week I don't have the time to do many outside projects. The couple that I do take on don't share any of the libraries used by employer.

AdamSpeight2008, on 28 December 2012 - 03:16 PM, said:

tlhIn`toq
Once you define a interface eg IFooBar and publicly release it, you should never change it. As it'll break existing code.

By changing the interface, are you saying change the actual signature of it? Change it from taking 2 strings to taking 2 strings AND an int? - Then yes, that will break programs.

I don't do much with making my own interfaces. I stick with proper methods mostly. Maybe interfaces would be better but we all get into habits and styles of coding. If one follows proper black-box design changing the internal functionality of a {library} method shouldn't break a program. A method only knows the parameters coming in and the return value, and causes no side affects. If for example I improve a method to merge 3 bitmaps so it is faster then I can safely replace the old code. Maybe I update the method that validates a string as a proper email address: It still returns true/false but accounts for odd/foreign cases better - that doesn't break one's program. Updating the method that returns the version of the OS to include Win8 isn't going to break an application.

[AdamSpeight2008]

It will also break if you add, remove or change any of the methods of the interface.
For example

Interface IFoo
  Function GetFoo() As Foo
End Function

is change to this, it'll break existing code because the definition has changed.

Interface IFoo
  Function GetFoo() As Foo
  Function GetManyFoos(HowMany As Integer) As Foo() 
End Function

Instead define a new interface

Interface IFoo2
 Inherits IFoo
  Function GetManyFoos(HowMany As Integer) As Foo() 
End Function

This post has been edited by AdamSpeight2008: 30 December 2012 - 03:33 PM

[stackoverflow]

It depends how large the projects are. I certainly don't separate everything for toy projects or test apps. I do, however, write in a comment that says something like, "This is bad practice in a large app ... etc...". I make it known what I am doing is not good practice but done for a specific reason.

[DaneAU]

I think its quite subjective how you do this. I personally would as a result of the relatively small size of your codebase simply create separate repositories.