Moving to Continuous integration and Continuous Delivery CICD

All companies (Big/small/startup) want to adopt to CICD because it helps to accelerated time to market (step ahead of the competition), improved productivity and efficiency and improved customer satisfaction and so on. New organizations have the benefit of implementing CICD from scratch. Existing companies have to drive steadily from manual/semi-automated to highly automated development. 

Continuous Delivery is a modern software development/release practices and principles aimed at building, testing and releasing software faster and more frequently. Building a successful CD pipeline means creating a DevOps culture of collaboration among the various teams involved in software delivery (developers, operations, quality assurance teams, management, etc.), as well as reducing the cost, time, and risk of delivering software changes by allowing for more incremental updates to applications in production.

Challenges:

Identifying and mitigating any challenges early in the CICD implementation will help you to head start the project as well as help you to address future challenges.  Most common challenges of enterprise:

Process Change: Most of the company has their own change control process handoff, Signoff, change control approval process and it takes its own time to approve.  Identify the delayed process and come up with acceptable solutions and fast forward solutions.

Technical Challenges:  For CICD, we will be using more third party tools and APIs. Avoid vendor lock-in for third party software, harden the security of open source software, and develop widely accepting standards and procedures for API and plugins.  Automation process needs its own testing and error handling properly to avoid any quality degradation in the pipeline. Frequent script changes are error-prone and need to be avoided. 

Organization change:  Organization need to make cultural change of Agile CICD model across the company.  Teams would have worked independently and worked on own timeline but with new Agile CICD model collaborative approach needs to be engaged in defined timeline.  The leadership team need to restructure the organization to break down barriers among teams and promote a collaborative culture.

Automation of CICD pipeline: (Need not to be same and depends on the organization requirement).

Ø  Maintain a Single Source Repository.     

Ø  Automate the Build                                       

Ø  Make Your Build Self-Testing

Ø  Everyone Commits To the Mainline Every Day

Ø  Every Commit Should Build the Mainline on an Integration Machine

Ø  Deployment to integration environment

Ø  Integration tests

Ø  Scenario/regression tests

Ø  Deployments to UAT and Performance test environment

Ø  More scenario/regression tests

Ø  Performance tests

Ø  Alerts, reports and Release Notes sent out

Ø  Deployment to release repository

Ø  Deploy to production

Source Control Management: Source control management tools provide the ability to store separate versions of things, including the ability to manage variation through branching and merging.

Tools: Git, Subversion, Mercurial, Veracity, TFS, ISPW

Continuous Integration: integration is a software development practice where developers regularly merge their code changes into a central repository, after which automated builds and tests are run.

Tools: Jenkins, codeship, Teamcity,continuum, Travis CI, CircleCI, Bamboo, Visual Studio, shippable

Build automation: Build automation tools are software applications that automate the compilation and integration of executable software components from source code.

Tools: Maven, Gradle, Ant, Gulp, Cmake, MSbuild

Artifact Repository: Artifact repository tools are used to store and manage binary objects that are used to build software.

Tools: Dockerhub, npm, Nexus, Artifactory

Continuous delivery: Continuous Delivery simply builds on this foundation and extends it more into the domain of deployment and project delivery.

Tools: Jmeter, Cucumber,Junit,Mocha,Selenium,Qunit,TestING, Jasmine, Karma,Qunit.LoadRunner, WebLoad, Testing Anywhere

Configuration & provisioning:  Configuration management tool automates deployment of just about any infrastructure and application stack used to create modern cloud and enterprise IT.

Tools: Chef, Ansible, Salt,CFengine, puppet, Terraform, vagrant

Application Security: Application security solutions need to be naturally integrated into the CICD workflow. 

Tools:  HPE Fortify, Tripwire, Snort

Application logging:  Application logging system helps to troubleshoot application problems and investigate security incidents in minutes instead of hours or days, avoid service degradation or outages, deliver compliance at lower cost and gain new business insights.

Tools: Logstash, Splunk, SumoLogic, Logentries

Application monitoring: Application monitoring needed for the entire system to ensure systems, applications, services, and business processes are functioning properly. In the event of a failure, system can alert technical staff of the problem, allowing them to begin remediation processes before outages affect business processes, end-users, or customers.

Tools: Nagios, Zabbix, NewRelic, Appdynamics, Datadog, Dynatrace

Closing statement:

Identifying the common challenges to put CICD into practice, evaluate the situation should be a first preparatory step in your implementation. Gain an accurate picture of the current baseline process, structure of your implementation in measurable phases, and then work on dealing with these challenges to clear the way for your first delivery pipelines.   Integration tools tie together from multiple vendors into a single integrated tool chain, ensuring data consistency and integrity is important. Don’t need to automate the entire system at once, implement automation in phased approach. Good luck with your CICD Move!

Moving to Cloud computing (Legacy application migration to cloud)

Huge momentum has built to migrate applications to cloud computing nowadays. There are several motivation factors to adapt to cloud:  cost savings IT infrastructure and maintenance, time to market, rapid deployment and provisioning, auto-scaling, flexible capacity, and business continuity.

Moving existing applications is not simply a matter of lifting and shifting to a cloud platform. Migration of existing applications to cloud computing must be done in a strategic and systematic manner. Identify individual business problems with existing applications that cloud computing can potentially address and provide specific business justification that the cloud is the right strategic alternative.

Enterprise applications must be thoroughly assessed to determine which application can benefit most from early migration to the cloud. Key considerations including costs of migration, application redesign, application performance and availability, security and privacy requirements, and regulatory requirements must be taken into account.  Create a pilot for one or two of these types of applications, test thoroughly, and gather customer feedback so that trust and confident can be built before going live.

The important point is that decisions about moving to the cloud need to start from an application standpoint, with a thorough consideration of followings:

Stateless: The thump of rule is keep your application from being affected by dynamic scaling: Build your application to be as generic and stateless as possible.  Avoid singleton.

Hardware dependency: Services that application calls should not be pointing particular host names or IP addresses. Abstracting environment-specific dependencies into a set of property files is still an inadequate. A better approach is to use an external service registry to resolve service endpoints like load balancer with virtual name.

Avoid local file system:  Avoid local file system as a store for temporary information, put temporary information in a remote store such as an S3, Amazon EFS, RDS or NoSQL database.

State-fullness: HTTPSession state is often stored in-memory, which presents a problem if your entire application server can be added or removed or moved at any time in the cloud. The best practice is to push it out to a highly available store that is external to your application server; that is, put it in a distributed caching store, such as  Memcached, or Redis.

Application Log Fie:  Application services are in auto scaling added/moved/removed often and storing log file in the local file system will crash your debugging option or the file you are looking for may not be available for debug. Use an open source aggregator, such as Scribe or Apache Flume, ELK or a commercial product, such as Splunk.

Asynchronous Calls: Consider that asynchronous protocols (Queue & RESTAPI) are still alive and well and can be extremely effective for cloud application programming.

Portability: Avoid OS dependency and application should be portable to any time of middleware.  Use Cygwin and Mono to port between Linux/.Net.

Closing statement:

Management should realize that every significant transition in technology has resulted in a significant rewrite or re-implementation of the technologies that businesses depend on but deep analysis and systematic approach of changing to service-oriented architecture (SOA) can make your transition smooth and success.