Software Development Automation — Part I
Developing software requires many repetitive tasks. No matter if you rebuild/redeploy an app, restart services or reset an application state, all these tasks add up a substantial amount of non-productive time. There is no secret that the best way to improve productivity is automation.
For example, using docker to build your application and bundle app services with docker-compose would provide an excellent level of automation. But even in that case, there are some remaining tasks: to trigger a build, restart services or reset application state.
Some basic automation techniques may utilize bash aliasing or scripting.
The first one allows you to set a shortcut command for longer or bundled commands. For example, you could define the build app alias as:
The later allows you to write a script to execute shell commands automatically.
While these are viable automation options, I would focus on an automation framework that works with any stack allowing you to build and deploy an app with Docker, Kubernetes on developer machine or serverless abstracting security layer. While the other scripting automation tools use a more functional approach, “endly” framework is declarative, which is language agnostic. This framework allows you to define reusable workflows to automate typical software development tasks.
Security
To interact with specific application dependencies like private git repository, databases, or cloud services, you have to deal with credentials. Storing or hard coding credentials in plain text in automation scripts pose a severe security risk. Fortunately, endly stores all credentials encrypted, what’s more, using them in a shell comes with a high level of sensitive data obfuscation.
For example, when your application uses golang SDK with private git dependencies you can use the following build task:
The first thing in the build task enables git authentication prompt with a GIT_TERMINAL_PROMPT environment variable. During the build, any private git dependency would ask for respective credentials. Automation runner, in this case, would send expanded credentials to the terminal input, at the same time leaving an obfuscated expression in execution output.
Build and Deployment
During app development, you may need to rebuild and redeploy your project many times. Depending on your stack and application type, you may choose to build/deploy it in your machine, cloud, with a docker or in hybrid mode.
In the Dev-ops world, using docker with Continuous Integration automation would be the primary choice. However, when developing an application, any method optimizing developer workload should be explored. For example, if your application uses local snapshot dependencies, it may be way faster to build it on your machine than with a docker.
Developer machine
Developer machine is usually the most convenient way for debugging and developing an application. Ideally, you would want all your local changes to get redeployed many times for testing, before committing code. While this may be the fastest method to get your application out, it is also the least reliable, since you may have local modifications, or snapshot dependencies.
As part of automation endly has ability to set SDK with the requested version before the actual build takes place. On top of that, when SDK is missing on the target machine, it will install it for you.
For example you can build your react app on local or remote machine with the following workflow:
In the build workflow, you specify the target machine, followed by setting SDK and actual build instruction, finally providing command for starting an app and input for killing it.
More complex build scenario can be also easily implemented with the automation runner, take an example a java web application deployed to tomcat with the following workflow.
To deploy your web application to tomcat, you have to build it first with Java SDK and Maven build tool. Automation runner can automatically set and install the specified build and app container version. Once everything is in place, the remaining tasks are actual app build and deployment to the tomcat web apps folder.
Docker
Docker becomes a standard way of building and deploying applications.
Since it decouples applications from the underlying hardware, you can run your application and its services virtually anywhere. On top of that, it provides the most reliable way of building and deploying software.
For example, you can build your application with a multi-stage docker file. In the first transient stage, an app binary is created, followed by a final size optimized image build.
To integrate docker build and deployment with automation runner you can simply use the following workflow.
Hybrid Mode
In this mode, you create an application binary directly on your machine in the fastest possible way, followed by the most compacted docker image build with already built binary at hand.
For example to build and deploy the app in the hybrid mode you can use the following workflow:
Serverless
Function as a service
Function as a service (FaaS) allows you to develop, run, and manage application functionalities without having to manage the underlying infrastructure. You just deploy a piece of code as a function, which would automatically scale up or down depending on usage pattern.
Automation of serverless stack with all resources, permissions, and actual code provides excellent development optimization. In that case, you would work on a separated from the production environment, that enables you to rebuild and redeploy your serverless code as many time as needed before committing the code.
AWS Lambda
For example to build and deploy lambda function with dedicated role you can use the following workflow:
GCP Cloud Functions
Take another example how to build and deploy cloud function with the following workflow.
Wrapping Up
In this article, I’ve presented various cross-stack automation techniques streamlining the development process. They can also be used to automate development or staging environment setup and build foundation for automated end to end testing. While in this part, I’ve focused on build and deployment in the next, I would show how to automate application state setup. All presented automation workflows are available in Software Development Endly Automation Workflow Templates public git repository.