12 Replies - 2467 Views - Last Post: 11 June 2019 - 08:18 PM

#1 Skydiver   User is offline

  • Code herder
  • member icon

Reputation: 7056
  • View blog
  • Posts: 23,995
  • Joined: 05-May 12

What happened to the definition of DevOps?

Post icon  Posted 24 May 2019 - 07:21 PM

I recall many years ago when the term DevOps was coined, the idea was to remove the wall separating development and operations so that developers didn't just toss code over the wall over to the operations engineers. Every dev was now supposed to be treated like an operations person, and every ops person was supposed to be treated like a dev. Each could twiddle whatever they wanted to on the development or operations side so as to make smooth continuous delivery of software into production, and any production problems would be readily visible to developers. Everybody was now supposed to be a DevOps Engineer, and nobody was just a dev or just an ops engineer anymore.

Over the past few months reading various "DevOps" articles, (as well as seeing the internal implementation of so-called DevOps in our company's so called Agile environment), it feels more and more like the wall between development and operations is being raised again. Job listings saying "DevOps" is essentially looking for build engineers and sysadmins. They don't really want somebody who writes code. It feels like "DevOps" is just the new name for the Operations Engineers, and development is once again in its own silo.

What happened and why?

Is Sarbanes-Oxley and COBITT requirements imposing the separation of responsibilities in large companies/enterprises ("devs should never know production database passwords, only the ops guys doing the deployments should know that") not compatible with the small business/start up "everybody wears many hats -- what is this separation of responsibilities you are talking about?"

Or is it just institutional distrust between the two sides kick in: "I'm not going to let the devs touch the production machines. Are you crazy?" "I'm not going to let the ops guys change any of these stored procedures or Angular code. Are you insane?" The big Kumbaya was just a moment in time rather than an change in the way things are done?

Or is it that people just fixate on tools and processes related to DevOps and forgot all about the culture change that was the impetus for the creation of more streamlined tools and proceses?

Is This A Good Question/Topic? 2
  • +

Replies To: What happened to the definition of DevOps?

#2 jon.kiparsky   User is online

  • Beginner
  • member icon


Reputation: 11652
  • View blog
  • Posts: 19,806
  • Joined: 19-March 11

Re: What happened to the definition of DevOps?

Posted 24 May 2019 - 08:45 PM

If I had to guess, I might start by throwing out the suggestion that while maybe it's reasonable to expect someone to be good at writing code to deliver user-facing functionality and also understand and deeply sympathize with the tooling used to provide that functionality, it turns out to be really difficult to think about both sides of that equation at the same time, and productivity actually suffers when engineers try to do both at once.
Now, the stupid response to this would be to say "okay, everyone get into your silo and stay there" - so of course that's the thing most organizations end up doing, because it's the easiest thing to think of.
Was This Post Helpful? 0
  • +
  • -

#3 baavgai   User is online

  • Dreaming Coder
  • member icon


Reputation: 7485
  • View blog
  • Posts: 15,514
  • Joined: 16-October 07

Re: What happened to the definition of DevOps?

Posted 25 May 2019 - 04:24 AM

Interesting. I didn't know the history of it, but I suppose by your definition, I am a devops guy. I write the programs, but I'm technically the DBA, and I effectively wear all the hats.

Personally, I see operations as one of those things you can "pick up" if you're of the right mindset. Every process can be googled and discovered recipes documented and implemented. Anyone working on a given system long enough will find themselves treading sysadmin waters at some point.

Developer is a different animal. Sure, you can google all the recipes, but building something new requires both a foundation skill set and the (rare) ability to think like a programmer. A developer can pick up the operations side without issue, but is the converse true?

As skill sets go, neither side much cares for the other. Even if you're good at both, one will probably be more of a chore than the other. And, again, frankly, good sysadmins are not good coders by default. And most I've run into don't want to be.

My short take is that "DevOps" is one of those suit level pipe dreams like low code development. There's always the new business strategy that will maximize human resources in some manner that those resources really don't want to be maximized. Such ideas can succeed at ground zero deployment, but the farther away those ideas move from the initial zealots, the more fragile they will show themselves to be. And, the more dilute the idea will become.

Think of any industry paradigm you know well, XP, Agile, scrum, etc. Then compare it to examples you've seen in the wild. Hell, I couldn't even tell you what MVC means in any given domain. SoC, Saas, MVVM...
Was This Post Helpful? 1
  • +
  • -

#4 h4nnib4l   User is offline

  • The Noid
  • member icon

Reputation: 1434
  • View blog
  • Posts: 2,037
  • Joined: 24-August 11

Re: What happened to the definition of DevOps?

Posted 10 June 2019 - 02:29 PM

View PostSkydiver, on 24 May 2019 - 08:21 PM, said:

Or is it that people just fixate on tools and processes related to DevOps and forgot all about the culture change that was the impetus for the creation of more streamlined tools and proceses?


I think this is definitely at least part of the problem, and I think some of the tools themselves exacerbate the problem (like Microsoft's rebranding of TFS to Azure DevOps). It seems like a tempting pitfall for managers to think that if their teams are using Azure DevOps, they are obviously doing DevOps, even if everything is still in silos. We've been pushing for a more DevOps-inspired model for delivering solutions, but a lot of our devs just took that as meaning they shouldn't have to talk to anyone else before deploying to production.

To your point, I think the idea is that they shouldn't have to go to a separate group for deployments, because the app team is cross-functional and able to responsibly and effectively manage both the CI and the CD. At least that's how we've envisioned it. Unfortunately, our reality is that we don't even have good testing practices (beyond half-hearted UAT), or dedicated QA for that matter, so it feels a bit like trying to teach a J-turn to someone still learning a left turn...
Was This Post Helpful? 1
  • +
  • -

#5 DarenR   User is offline

  • D.I.C Lover

Reputation: 634
  • View blog
  • Posts: 4,209
  • Joined: 12-January 10

Re: What happened to the definition of DevOps?

Posted 11 June 2019 - 05:02 AM

i was a devops in my last position and now a software engineer in my new company.

the difference is my devops position only built, deployed and maintained code as well as maintaining servers and their functions. the software engineer, i make code, test, build it and release it not so much of the server maintenance.

i think it is the Agile crap that is changing the functions of commonly known names. btw im am not a fan of the agile method since it seems to introduce more bugs then it fixes.
Was This Post Helpful? 1
  • +
  • -

#6 Skydiver   User is offline

  • Code herder
  • member icon

Reputation: 7056
  • View blog
  • Posts: 23,995
  • Joined: 05-May 12

Re: What happened to the definition of DevOps?

Posted 11 June 2019 - 11:15 AM

Part of the unspoken "features" of Agile is fail fast to get feedback faster. Unfortunately, the "fail fast" is practically equivalent to bugs when they make it all the way to production and is missed during the sprint review with the customer (assuming they are doing scrum).
Was This Post Helpful? 1
  • +
  • -

#7 jon.kiparsky   User is online

  • Beginner
  • member icon


Reputation: 11652
  • View blog
  • Posts: 19,806
  • Joined: 19-March 11

Re: What happened to the definition of DevOps?

Posted 11 June 2019 - 12:03 PM

Not sure how this got on to Agile, but: I think one of the problems with Agile is that there's a lot of ways to do it, and many of them are wrong. People often do a sort of "modified" process because they don't want to commit to the hard stuff, but the hard stuff is where you get the long-term wins. We've been using a strict Scrum process on my team for a while, and while it takes a certain amount of maintenance we're doing really well with it now.

But again, if you half-ass it, it's probably not going to work.

> "fail fast" is practically equivalent to bugs

I'm not sure I understand that. What do you mean when you say "fail fast"?


On-topic: We have recently dedicated one guy, who really likes dev ops stuff, to working almost entirely on dev ops. That's working better than when we tried to have everyone doing dev ops tickets.
Was This Post Helpful? 0
  • +
  • -

#8 Skydiver   User is offline

  • Code herder
  • member icon

Reputation: 7056
  • View blog
  • Posts: 23,995
  • Joined: 05-May 12

Re: What happened to the definition of DevOps?

Posted 11 June 2019 - 03:32 PM

"Fail fast" as in quickly produce what you think the customer wants as quickly as possible (without killing yourself or your team). Get that build in front of the customer as early as possible while it is still in testing so that they can:
  • Yes! That's exactly what we had in mind.
  • No! You we must have understood each other.
  • We like it, but can you change X, Y, and Z?

The latter two are the "Fail fast" mode where the code has not yet even gone on to formal SIT or UAT, but you have quickly gotten feedback that you are headed down the wrong path.
Was This Post Helpful? 0
  • +
  • -

#9 Skydiver   User is offline

  • Code herder
  • member icon

Reputation: 7056
  • View blog
  • Posts: 23,995
  • Joined: 05-May 12

Re: What happened to the definition of DevOps?

Posted 11 June 2019 - 03:38 PM

The Agile Manifesto

Quote

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.


To me "fail fast" is reflected by the first 4 principles behind the manifesto:

Quote

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Business people and developers must work together daily throughout the project.

Was This Post Helpful? 0
  • +
  • -

#10 Skydiver   User is offline

  • Code herder
  • member icon

Reputation: 7056
  • View blog
  • Posts: 23,995
  • Joined: 05-May 12

Re: What happened to the definition of DevOps?

Posted 11 June 2019 - 03:55 PM

And sort of to tie this back into the DevOps discussion, my understanding of the early origins of DevOps is that dev teams originally managed to break out of the waterfall when they write and test their code, but they still had to ride the waterfall process when it was time to go into production. So DevOps was supposed to provides ways of doing continuous delivery so that the last few stages of going into production is as agile as the development process.

(In my current company, our change management process still assumes a waterfall process. They are starting to get snowed under by teams trying to deliver every two weeks. Their response: add more roadblocks and hoops to jump through before they approve the change going into production. Continuous delivery this is not.)
Was This Post Helpful? 0
  • +
  • -

#11 jon.kiparsky   User is online

  • Beginner
  • member icon


Reputation: 11652
  • View blog
  • Posts: 19,806
  • Joined: 19-March 11

Re: What happened to the definition of DevOps?

Posted 11 June 2019 - 04:10 PM

View PostSkydiver, on 11 June 2019 - 05:32 PM, said:

"Fail fast" as in quickly produce what you think the customer wants as quickly as possible (without killing yourself or your team). Get that build in front of the customer as early as possible while it is still in testing so that they can:
  • Yes! That's exactly what we had in mind.
  • No! You we must have understood each other.
  • We like it, but can you change X, Y, and Z?

The latter two are the "Fail fast" mode where the code has not yet even gone on to formal SIT or UAT, but you have quickly gotten feedback that you are headed down the wrong path.


Okay, I'm still not sure I'm following.

So yes, in an Agile approach you typically want to deliver small chunks of useful product often and get feedback on it as early as possible. But how does that imply more bugs?
When we do this, it's usually a way to find bugs, not create them.

Unless you mean people are putting code in front of the end user before it's been tested, like deploying unfinished work? I agree that that would be wrong but I don't think it would be something Agile recommends. Typically you'd be putting the build in front of the "customer" in the sense of some representative stakeholder, not shipping to the end user.
Was This Post Helpful? 0
  • +
  • -

#12 Skydiver   User is offline

  • Code herder
  • member icon

Reputation: 7056
  • View blog
  • Posts: 23,995
  • Joined: 05-May 12

Re: What happened to the definition of DevOps?

Posted 11 June 2019 - 05:16 PM

Unfortunately, from what I've observed, people are not doing Agile correctly. They are not working with the business everyday. Once the story has been put into the backlog, the next time the business user hears about the story is when it's been accepted to go into a sprint, and then again another blackhole until 2 days before the end of the sprint when they are asked to approve the change because testing believes that all the story's acceptance criteria. The business is compelled to say "Yes." even though they never actually get to play with the build. The time when they'll actually get to play with the build is when it gets deployed to production. That's the time when the bugs actually get uncovered. So the "fail fast" that was supposed to happen within hours or days of the developer having finished implementing the story, gets bypassed. The "fail fast" happens within hours or days of production deployment.

Part of this is office politics because the dev managers/leads don't want to look bad, so they insist that testing get first crack at the new feature before the business sees it -- supposedly so that the business doesn't waste their time. Of course this adds to the pile of work that testing needs to do because they want to do full integration tests when all of the stories for the sprint have come in, not stories as they come hot off the continuous integration build server. So what I'm seeing is that all the testing gets compressed to the end, where business effectively just sees a quick demo of the stories implemented for the sprint, but they never get to actually play with the build in a freeform hands on environment.

As you earlier pointed out, it's because agile is (often) implemented/adopted half-assed. People need to realize that Agile requires both development and the business collaborating everyday. It's not just the business have a more direct line to the requirements/features and their prioritization. The direct line also needs to go the other direction where development can impose on the business' time and have them provide daily feedback, and perform active testing.
Was This Post Helpful? 0
  • +
  • -

#13 jon.kiparsky   User is online

  • Beginner
  • member icon


Reputation: 11652
  • View blog
  • Posts: 19,806
  • Joined: 19-March 11

Re: What happened to the definition of DevOps?

Posted 11 June 2019 - 08:18 PM

Quote

Unfortunately, from what I've observed, people are not doing Agile correctly.


Yeah, I think we're in agreement there. I'm lucky enough to have a workplace where engineering is pretty well integrated with the business, which helps a lot. I guess some businesses try to maintain a sort of separation between engineering and the rest of the enterprise (as you put it, "the business") perhaps on the theory that because engineers are expensive they shouldn't be spending their time understanding what the actual company is doing. That seems weird to me - but that's not entirely an Agile thing, it's more of a thing about the organization of an enterprise.

We typically deploy features when they're done, which for us means two engineers have done code review and product has hammered on it on a staging server. Bugs do go out, it's true, but that's really a matter of priorities. If we wanted to invest more in QA, we could, and we'd ship fewer bugs, but overall we've generally bet on ship it fast and fix it fast instead of the mills-of-god approach. The hope there, of course, is that bugs will become obvious to us. A subtle logic glitch which doesn't provoke any visible error on the server might go undetected for a while - but on the other hand, developing the QA tests to catch those issues is extremely expensive and it's hard to know how far you need to go with that in the first place. If it gets by two engineers and our product guys, it's already an extremely lucky bug. How much do we have to invest in QA to be luckier than the bug?

This is actually a place where I get to toot the clean-code horn, and I do this in pull requests all the time. I'd say that about 80% of my review comments are about style issues - unclear naming, functions too long, unclear expression of intent, duplicated code - and by hammering on that stuff over and over again I've helped to build a team that generally writes clean code the first time - if only because they know that they're going to have to do it right eventually if I pull the review :)
And the nice thing about clean code is, it's really easy to find subtle logic errors if you've mowed down the tall weeds that they like to hide in.
Was This Post Helpful? 2
  • +
  • -

Page 1 of 1