GitLab Flow #

Starting out with DevSecOps requires a well-thought-out workflow, but that can sometimes seem like a daunting challenge. Luckily, there is GitLab Flow that can help. GitLab Flow is a prescribed approach to help organizations successfully apply DevSecOps processes. In this article, we delve into how GitLab Flow can be used together to help organizations be successful with DevSecOps.

What is GitLab Flow? #

GitLab Flow is a prescribed and opinionated end-to-end workflow for the development lifecycle of applications when using GitLab, a DevSecOps platform with a single user interface and a single data model. GitLab Flow is based on best practices and lessons learned from customer feedback and our dogfooding. Furthermore, GitLab Flow spans across the stages of the DevSecOps lifecycle, forming an efficient workflow with an inner feedback loop for reviewing a specific update and an outer feedback loop for improving the entire application, as well as the development lifecycle itself.

As you can see by the many stages in GitLab Flow, there is much more to developing software than writing code.

Planning #

The first portion of GitLab Flow is planning, which sits on the outer feedback loop of GitLab Flow. It encompasses issues, merge requests, epics, milestones, iterations, release, release evidence, and more. Let’s cover what roles these components play in GitLab Flow.

Issues #

Issues are where product problems or new features are defined and where team members can collaborate. As an issue is created, you can populate its title and description. Many stakeholders can participate in comment threads on an issue.

Issues can be organized and visualized in issue boards, which are a software project management tool that can be used as kanban or Scrum boards. These boards help teams plan, organize, and visualize a workflow for a feature or product release. Different categories of boards can be created and issues can be moved from one board to another one with a simple drag and drop.

Merge Requests #

Merge requests are where solutions are developed. As release components, issues and merge requests provide the auditability and tracking of application changes done by stakeholders, such as DevOps and platform engineers, system and database administrators, security engineers, and developers. In addition, issues and merge requests are key inputs for the release planning process.

Merge requests can be individually created or created from an issue. Creating a merge request from an issue automatically relates it to that issue so when the merge request is merged its associated issue is automatically closed. Merge requests can also be manually related to an issue.

Issues with the same theme can be grouped together in an epic to organize the work to be done. Epics can have child issues and sub-epics and/or be linked to epics across the organization. Iterations can be used to track sprints of work, and can be manually scheduled or scheduled automatically using GitLab iteration cadences to streamline planning workflows. In addition, iterations include burndown and burnup charts. Burndown charts help track overall progress towards a project’s total scope, while burnup charts track the daily total count and weight of issues added to and completed in a given timebox.

Milestones #

Teams can use milestones to organize issues and merge requests into a cohesive group with an optional start date and an optional due date. Milestones are typically used to track releases and can track issues and merge requests at a project level or group level. Similar to iterations, milestones also provide burndown and burnup charts to show progress.

Milestones can be associated with a release, whose automated creation generates many artifacts, including the release evidence. The release evidence is an automatically collected snapshot of data that’s related to the release. In addition to test artifacts and linked milestones, job artifacts can optionally be included in the release evidence, which can facilitate internal processes such as external audits.

Epics, milestones, and iterations can be visualized via the Roadmaps page, which helps track release progress and streamline the release process.

Once the planning takes place, the work towards the resolution of a problem or a new feature can start. This happens in merge requests. Let’s delve deeper into how that happens in GitLab Flow.

Merge requests and pushing code #

The second portion of GitLab Flow is related to merge requests and pushing code. As mentioned earlier, merge requests are where solutions are developed through collaboration among stakeholders across the organization. This collaboration can happen in a distributed manner and asynchronously. Participants can take advantage of collaborative capabilities, such as tagging, inline suggestions, inline comments, merge request comments, review threads, and review requests, which can help improve code quality, availability, reliability, and performance. Right after the creation of the merge request is the start of the GitLab Flow inner feedback loop, which is where code and fix pushes, test and scan runs, and collaboration and update reviews take place.

Pipelines #

As updates are applied to a feature branch via merge requests, pipelines — if defined — are automatically executed. Pipelines can have multiple stages and jobs to build and test, and then deploy the application or microservice to a review environment. In that review environment, the updates can be dynamically verified before they are merged to the main branch. This automation helps streamline the application update and review processes.

While pipelines execute on branch updates, they can include automated tests and scans, which helps in shifting security left.

Shifting security left #

The third portion of GitLab Flow is shifting security left, which is also part of the GitLab Flow inner feedback loop.

In addition to DevOps and platform engineers, system and database administrators, and developers, some of the stakeholders collaborating in a merge request may be concerned about security and compliance, which is where automated tests and security scans play a role. Scans can be simply included in a pipeline via readily available templates and/or can be automatically executed within a merge request pipeline. GitLab provides a broad set of built-in security scanners and analyzers that can be leveraged by GitLab Flow, but the DevSecOps platform can also accommodate third-party and custom scanners.

GitLab Flow shifts security left in the pipeline to detect and resolve defects as early as possible in the software development process. It is much simpler and cheaper to fix vulnerabilities early in the development cycle than once the application is in production, where an unscheduled outage can affect your users and revenue.

The built-in security scanners and analyzers provided by GitLab include: unit testing, infrastructure-as-code (IaC) scanning, static application security testing (SAST) scanners, dependency scanning, secret detection, container scanning, API security, web API fuzz testing, and coverage-guided fuzz testing. In addition, GitLab provides a variety of security dashboards and reports to manage and visualize vulnerabilities, such as the Dependencies list, Security dashboard, Vulnerability Report, and vulnerability pages.

Besides SAST scanners, GitLab provides dynamic application security testing (DAST) scanners, which require a running application. When leveraging these scanners, GitLab is capable of automatically provisioning a DAST environment for the DAST scans and then performing a complete cleanup of all resources post-DAST testing. In addition, for running containers, GitLab provides operational container scanning, which scans container images in your cluster for security vulnerabilities.

The scans mentioned above can be executed automatically within a merge request pipeline or, in some cases, can be scheduled for execution via scan execution and merge request approval policies. These policies can be defined via the GitLab UI or YAML files and are configured in a separate project, allowing segregation of duties for reusability, maintenance, and management. Scan execution policies require that security scans be run on a specified schedule or with the project pipeline, and merge request approval policies take action based on scan results. Security engineers or teams can define these policies to enforce security processes across the organization and GitLab Flow may encounter or leverage these as it spans through its steps.

To enforce security and compliance across projects in your organization, you can use compliance labels and pipelines. Compliance labels and pipelines can be made mandatory to execute before a project’s own pipeline. With this approach, you can ensure that all teams within your organization meet your security and compliance standards. In addition, you can secure your applications against cyber attacks, conform to government compliance standards, and always be audit-ready.

The main goal of all of these GitLab Flow security prescriptions is to fix vulnerabilities early in the development cycle rather than once the application is in production, where remediating a vulnerability can prove to be very costly in reputation and revenue.

As vulnerabilities are mitigated within the GitLab Flow inner feedback loop and more updates are applied to the application in the feature branch, stakeholders need to re-review these updates to ensure that the updates have taken place and no regressions have inadvertently been introduced.

Continuous review #

The next portion of GitLab Flow is reviewing features, which prescribes the continuous review of applications. Reviewing features involves the ability to stand up a review environment to which the interim application (feature branch) is deployed so that stakeholders can review it in real time and provide feedback. The interim application can then be continuously adjusted until it is ready to be merged to the main branch. GitLab Flow also prescribes the cleanup of all provisioned review environment resources at the moment when the merge request is merged to the main branch.

This iterative automated review process is part of the inner feedback loop in GitLab Flow. The GitLab Flow inner feedback loop terminates when all review items are addressed and the merge request is approved and merged to the main branch, which triggers the deployment of the application to production.

Deploying applications and infrastructure #

Depending on an organization’s needs, either continuous delivery or continuous deployment is prescribed by GitLab Flow. Whereas continuous delivery is the frequent release of code by triggering the deployments manually (e.g., to production), continuous deployment is the automated release of code (e.g., to production) without human intervention. Let’s cover continuous delivery first.

As you release your software using continuous delivery, you have a few deployment options. You can establish a freeze window and then deploy using advanced deployment techniques, such as canary, blue/green, timed, and incremental rollouts. Incremental rollouts can lower the risk of production outages delivering a better user experience and customer satisfaction. Advanced deployment techniques can also improve development and delivery efficiency, streamlining the release process.

As you release your software using continuous deployment, all changes/updates go directly to production. Progressive delivery approaches like feature flags, which allow you to separate the delivery of specific features from a launch, are a good way to reduce risk and manage what functionality to make available to production users. Feature flags support multiple programming languages and allow developer experimentation and controlled testing. You can even use feature flags to roll out features to specific users.

Although GitLab supports all these deployment approaches, GitLab Flow allows for the adoption of the approach that best fits the organization and/or specific project needs.

Monitoring applications and DevSecOps processes #

Once your application has been deployed to production, it needs to be continuously monitored to ensure its stability, performance, and availability. In addition, as the DevSecOps processes execute, they are measured, providing the opportunity to improve their performance and efficiency. The monitoring capabilities are provided by GitLab and, as such, can be leveraged by GitLab Flow.

For running containers, GitLab provides operational container scanning (OCS), which scans container images in your cluster for security vulnerabilities. These scans can be automated by scheduling them when to run and any found vulnerabilities are automatically displayed in a security dashboard. The OCS can help keep your cluster applications secure and preempt any cyber attacks that can lead to leaks of private data and even cause unexpected outages.

Error tracking allows developers to discover and view errors generated by their application. All errors generated by your application are displayed in the Error Tracking list in GitLab. Error tracking can help with availability and performance of your applications by detecting and resolving unexpected application conditions fast.

GitLab can accept alerts from any monitoring source, including Prometheus, via a webhook receiver. As alerts come in, they are displayed in the GitLab Alerts list, from which you can manually manage them. Alerts can also automatically trigger the creation of incidents, ChatOps, and email messages to appropriate individuals or groups. All these capabilities streamline the alert resolution and management process.

As incidents are created, due to production problems, they appear in the GitLab Incidents list for incident management. You can manage one or more incidents, sort them, search them, assign them, set their statuses, and even see their SLA preset countdown timer. Moreover, you can create on-call schedules and rotations, escalation policies, and set up paging and notifications to handle incidents. In addition, you can link an incident to an alert so that when the incident is closed, its associated alert is automatically resolved. Incident timelines are another capability for executives and external viewers to see what happened during an incident, and which steps were taken for it to be resolved. All these capabilities streamline the incident management process so that they can be resolved as quickly as possible.

Audit events track important events, including who performed the related action and when in GitLab. These events are displayed in the GitLab Audit Events list and provide, among others, the action that was taken on an object, who did it, and the date and time of its occurrence.

All the lists and dashboards mentioned above can help preempt out-of-compliance scenarios to avoid penalties as well as streamline audit processes. For your running applications, they generate the data and metrics that can be used in the GitLab Flow outer feedback loop to help improve and optimize your applications and lower the risk of unscheduled production outages.

Continuous improvement #

When applying GitLab Flow, you also have the opportunity to use the insight that GitLab provides in the form of end-to-end process metrics dashboards to continuously improve not just your application but also your software delivery performance. These dashboards and their metrics are auto-generated by GitLab and are always available.

The Value Stream Analytics dashboard #

You can track and monitor your application development lifecycle through the Value Stream Analytics Dashboard, where you can check project or group statistics over time. This dashboard is customizable but you can get started quickly by creating a value stream using a GitLab-provided default template. The default dashboard displays metrics for each of the pre-defined stages of your value stream analytics, namely Issue, Plan, Code, Test, Review, and Staging, as well as a graph with the average time to completion for each. It also shows the value stream analytics key metrics: lead time, cycle time, new issues, commits, and deploys. You can use these metrics to find areas of improvement in the stages of your value stream.

DORA metrics dashboard #

To view the performance metrics that measure the effectiveness of your organization’s development and delivery practices, GitLab provides the DORA (DevOps Research and Assessment) metrics dashboard, which displays four key metrics: Deployment Frequency, Lead Time for Changes, Time to Restore Service, and Change Failure Rate. Deployment Frequency measures how often your organization deploys code to production or releases it to end users. Lead Time for Changes measures how long it takes to go from code committed to code successfully running in production. Time to Restore Service measures the time needed to restore services to the level they were previously, in case of an incident. Finally, Change Failure Rate is the percentage of changes to production or released to users that resulted in a degraded service (for example, a change that caused a service impairment or outage) and subsequently required remediation (required a hotfix, rollback, patch). These four key metrics are outcomes of your current processes and give you the opportunity to improve the factors and capabilities that drive them.

Why use GitLab Flow? #

GitLab Flow is a prescribed approach, practiced by our customers and users worldwide, that can provide the following benefits:

• Higher productivity via the automation capabilities provided by GitLab and its single user interface and data model, all leveraged by GitLab Flow
• Accurate insights into the end-to-end DevSecOps lifecycle to support continuous improvement
• Built-in dashboards and metrics that can help you optimize your applications and DevSecOps processes
• Higher code quality and improved reliability and availability of your applications
• Better application security through built-in security scanners and capabilities
• Compliance- and audit-readiness via built-in compliance features
• Shorter cycle times that can help you increase deployment frequency
• Continuous review enabled by the GitLab Flow inner feedback loop
• The GitLab Flow inner feedback loop can help you optimize application updates leading to better code quality and higher reliability and availability of your applications
• The GitLab Flow outer feedback loop can help you improve your applications as well as the development lifecycle itself
• High levels of collaboration among stakeholders in your organization
• Shifting security left to help find vulnerabilities in applications before they make it to production to avoid costly, unscheduled outages
• Lower risk when deploying to production via the advanced deployment techniques and progressive delivery approaches supported by GitLab
• AI-powered capabilities that span across the entire development lifecycle and can boost productivity, code quality, continuous improvement, security and compliance, and more
• Support for cloud-native and non-cloud-native applications
• Multi-cloud support for hybrid/multi-cloud applications
• Shifting security left to help you find vulnerabilities in your applications before they make it to production so that you can avoid costly unscheduled outages

How can you get started with GitLab Flow? Leveraging GitLab Auto DevOps or parts of it is a good starting point for applying GitLab Flow principles to your application development lifecycle.

GitLab Flow and Auto DevOps #

Auto DevOps applies GitLab Flow throughout all its stages and jobs. You can think of it as a good example for the instantiation of GitLab Flow.

Auto DevOps is a collection of predefined, out-of-the-box CI/CD templates that auto-discover the source code you have. Based on best practices, these templates automatically detect, build, test, deploy, and monitor your applications.

The Auto DevOps pipeline shifts work left to find and prevent defects as early as possible in the software delivery process. The pipeline then deploys the application to staging for verification and then to production in an incremental/timed fashion.

Auto DevOps gets you started quickly, increasing developer productivity, and it can be easily customized to your needs, with support for the most common programming frameworks and languages. Auto DevOps is modular, customizable, and extensible, which allows you to leverage pieces of it in your pipelines or apply all of it for your application.

Source: https://about.gitlab.com/blog/2023/07/27/gitlab-flow-duo/#what-is-gitlab-flow%3F