Differences
This shows you the differences between two versions of the page.
|
ii:labs:s2:05 [2022/05/31 15:34] radu.ciobanu [Swarm networks] |
ii:labs:s2:05 [2025/04/13 18:19] (current) florin.stancu |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ~~NOTOC~~ | ~~NOTOC~~ | ||
| - | ====== Lab 05 - Docker Swarm ====== | + | ====== Lab 06 - Docker Compose ====== |
| ===== Objectives ===== | ===== Objectives ===== | ||
| - | * TODO | + | * Understand the YAML file syntax |
| - | * TODO | + | * Learn how to deploy multi-container applications using Docker Compose |
| - | * TODO | + | * Get familiar with Postman |
| - | * TODO | + | * Learn how to interact with a database |
| ===== Contents ===== | ===== Contents ===== | ||
| Line 17: | Line 17: | ||
| ===== Introduction ===== | ===== Introduction ===== | ||
| - | In this lab, we will make the transition from Docker Compose to Docker Swarm, **//the service orchestrator//** provided by Docker. Its role is to manage Docker services on one or more machines in a network (i.e., a cluster) of physical and/or virtual machines. Unlike Docker Compose, which runs containers on a single host, Docker Swarm runs //**services**// between multiple hosts. Like Compose, Docker Swarm uses YAML configuration files. | + | Normally, in order to run containers we need to execute the corresponding run command (**//docker run//**) and set all the necessary parameters. This process can be difficult and repetitive when we need to start multiple containers. One way to "save" the running configuration is to create scripts. The problem with running multiple scripts is the loss of uniformity in configuration (which container connects to which network, who communicates with whom, etc.). |
| - | ===== Docker Swarm architecture ===== | + | [[https://docs.docker.com/compose/|Docker Compose]] is a utility created by Docker that is used to centralise the configuration process of a container-based application in a declarative manner, using Yet Another Markup Language (YAML) configuration files. |
| - | In the image below (taken from the [[https://docs.docker.com/engine/swarm/how-swarm-mode-works/nodes/|official documentation]]), you can see the architecture of a Docker Swarm cluster. | + | Moreover, the format for Compose files is also used in **//Docker Swarm//**, the orchestrator created by Docker for managing Docker services. |
| - | {{:ii:labs:s2:swarm-diagram.png?direct&600|}} | + | <note tip>You will notice that in this lab we use the terms //**service**// and //**container**// interchangeably. This is because Docker Swarm works with services, while Docker Compose works with containers. We refer to both terms in the same context because the configuration is 90% identical, regardless of whether Swarm or Compose is used. </note> |
| - | Host machines that are part of a Swarm are called **//nodes//** and can have two roles: | + | ===== Installation ===== |
| - | * //**manager**// - administrative and functional role; maintains cluster consistency, launches services, exposes network endpoints | + | |
| - | * //**worker**// - functional role; runs services. | + | |
| - | Of all the manager nodes, only one node is the **//leader//**, which has the role of creating tasks and logging. The tasks are then distributed to the other nodes. | + | For Windows and MacOS, Docker Compose is part of the Docker Desktop installation. For Linux, the installation is done [[https://docs.docker.com/compose/install/|according to the official guide]]. |
| - | <note important>There must always be exactly one leader node.</note> | + | ===== Key items ===== |
| - | ===== Creating a Docker Swarm ===== | + | ==== YAML file format ==== |
| - | Once we have a cluster of machines running Docker, we can initialise a Docker Swarm. Thus, we can run the following command on the node that will be the leader (the option //**%%--%%advertise-addr**// is required when the node has several network interfaces and it must be specified which of them is advertised): | + | [[https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html|YAML]] files are generally used to write declarative configurations. The format is very easy to understand and employ, as follows: |
| - | <code bash> | + | * //**key:value**// elements are used |
| - | $ docker swarm init --advertise-addr 192.168.99.100 | + | * indented paragraphs are children properties of the previous paragraphs |
| + | * lists are delimited by //**-**//. | ||
| - | Swarm initialized: current node (qtyx0t5z275wp46wibcznx8g5) is now a manager. | + | ==== Docker Compose file example ==== |
| - | To add a worker to this swarm, run the following command: | + | |
| - | docker swarm join --token SWMTKN-1-4hd41nyin8kn1wx4bscnnt3e98xtlvyxw578qwxijw65jp1a3q-32rl6525xriofd5xmv0c1k5vj 192.168.99.100:2377 | + | |
| - | To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions. | + | |
| - | </code> | + | |
| - | <note tip>As you can see, the above command generates two more commands that we can use to add other nodes into the cluster as workers or as managers. </note> | + | <code yaml> |
| + | # docker-compose.yml | ||
| + | #version: "3.8" | ||
| - | We can verify that the swarm was created successfully by running the command below on the leader (where we have two nodes called //**node1**// and //**node2**//, the former being the leader, and the latter being the worker): | + | services: |
| + | api: | ||
| + | build: . # builds the image from a Dockerfile | ||
| + | image: register-image-name:version # uses an image from a registry | ||
| + | environment: | ||
| + | ENVIRONMENT_VARIABLE: value | ||
| + | ports: | ||
| + | - "5000:80" | ||
| + | networks: | ||
| + | - lab5-network | ||
| - | <code bash> | + | postgres: |
| - | $ docker node ls | + | image: postgres:12 |
| + | volumes: | ||
| + | - lab5-volume:/var/lib/postgresql/data | ||
| + | - ./scripts/init-db.sql:/docker-entrypoint-initdb.d/init-db.sql | ||
| + | networks: | ||
| + | - lab5-network | ||
| - | ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS | + | volumes: |
| - | qtyx0t5z275wp46wibcznx8g5 * node1 Ready Active Leader | + | lab5-volume: |
| - | 0xbb9al1kuvn0jcapxiqni29z node2 Ready Active | + | |
| - | </code> | + | |
| - | ==== Docker Swarm services and service stacks ==== | + | networks: |
| - | + | lab5-network: | |
| - | When we talk about deploying an application in Docker Swarm, we move from the notion of container to the notion of //**service**//. A Docker service is a collection of (one or more) tasks, and a task is a container. Therefore, a service consists of one or more identical containers. The service monitors the lifecycle of the containers, always trying to maintain the condition of the containers provided in the configuration. In other words, a service is a set of containers with //**orchestration**//. | + | |
| - | + | ||
| - | Furthermore, a //**service stack**// represents several such services grouped in the same namespace. We can view a service stack as a multi-service Docker application. The easiest way to define a service stack is through a Docker Compose file, as we saw in [[ii:labs:s2:04|lab 4]]. The behaviour of services in a stack is similar to that of Docker Compose containers, except that the naming policy is different. | + | |
| - | + | ||
| - | <note tip>Any entity created in a stack (service, volume, network, etc.) will be prefixed by **//STACK-NAME_//**.</note> | + | |
| - | + | ||
| - | Docker Swarm has access to a new collection of options in the Compose YAML file, which will be specified in the [[https://docs.docker.com/compose/compose-file/compose-file-v3/#deploy|deploy]] attribute of a service. Below, you can see a snippet of a Docker Compose file showing some of these new options: | + | |
| - | + | ||
| - | <code yaml> | + | |
| - | [...] | + | |
| - | services: | + | |
| - | web: | + | |
| - | image: myimage | + | |
| - | deploy: | + | |
| - | replicas: 4 | + | |
| - | resources: | + | |
| - | limits: | + | |
| - | cpus: "0.2" | + | |
| - | memory: 50M | + | |
| - | restart_policy: | + | |
| - | condition: on-failure | + | |
| - | [...] | + | |
| </code> | </code> | ||
| - | In the YAML file fragment above, we run a service called //**web**//, which has four copies. Thus, there will be four different containers running the //**myimage**// image, each of which can respond to requests for the //**web**// service, depending on the load. Also, each instance is limited to 20% CPU (on all cores) and 50 MB of RAM. Last but not least, a container of the //**web**// service is restarted as soon as it encounters an error (the ultimate goal is to have 4 copies of the container on the network at any time). | + | === Version === |
| - | ==== Swarm networks ==== | + | The //**version**// attribute describes what [[https://docs.docker.com/compose/compose-file/|functionalities]] will be loaded when running the Docker Compose utility. |
| - | Unlike classic Docker and Docker Compose, networks created in Swarm no longer use the bridge driver, but the **//overlay//** driver. An overlay network is a network that spans all nodes in a swarm. For this reason, the exposed public ports will be unique per network. Therefore, two 3000 ports from two different services that connect to the same overlay network cannot be exposed. | + | <note warning>The ''version'' field has been deprecated on modern docker compose versions.</note> |
| - | <note tip>Docker Swarm balances load at network level.</note> | + | === Services === |
| - | A service that has been deployed to a particular port will always have that port reserved, no matter which node the container or containers actually run on. The diagram below (taken from [[https://docs.docker.com/engine/swarm/ingress/|the official Docker documentation]]) shows a situation where we have a service called //**my-web**// published on port 8080 in a three-node cluster. It can be seen that if we connect to port 8080 from any node IP address in the cluster, we will be redirected to a container running the 8080 external port-specific service, regardless of the node it is running on. | + | The //**services**// attribute describes the services/containers that will run after the configuration is started by Compose. Each service represents a container that will have the name and configuration of the service. In the example above, the containers will be named //**api**// and //**postgres**//. The most important properties of //**services**// are the following: |
| - | {{:ii:labs:s2:network.png?direct&600|}} | + | * **//build//** - specifies the path where the Dockerfile the container will be built from is located |
| + | * **//image//** - specifies the name of the image used to run the container | ||
| + | * **//ports//** - a list of entries with the format "host_port:service_port", which specifies which ports are exposed and/or mapped | ||
| + | * **//volumes//** - a list of entries with the format "host_volume:service_path" where the volume mappings are specified; the same rules that apply to CLI commands are maintained here as well; "host_volume" can be a standard volume or a bind mount | ||
| + | * **//networks//** - the list of networks which the service/container belongs to | ||
| + | * **//environment//** - object with entries of type "service_variable_name: value" which injects the environment variables specified when running the service/container. | ||
| - | ==== Differences between Docker Swarm and Docker Compose ==== | + | <note important> The **//build//** and **//image//** attributes are mutually exclusive.</note> |
| - | There are several key differences between Docker Swarm and Compose in YAML configuration files: | + | === Volumes === |
| - | * because Swarm is running services over the network, the **//build//** keyword cannot be used; services must be run based on images that already exist in a registry | + | |
| - | * service stacks do not support **//.env//** files (unlike Docker Compose) | + | |
| - | * Docker Compose runs single-host containers, while Docker Swarm orchestrates multi-host services. | + | |
| - | ==== Starting a service stack in Docker Swarm ==== | + | The //**volumes**// attribute describes the volumes used in the configuration. Volumes are passed as objects. If we do not want to change the default configuration, the value is an empty field. |
| - | Once the Docker swarm has been created and initialised, the command to deploy a service stack is as follows (where the configuration is in the file //**my_stack.yml**// and the stack's name will be //**lab5**//): | + | <note tip>The top-level **//volumes//** property must be written at the same indentation level as **//services//** and it should not be confused with the child property **//volumes//** within the service configuration.</note> |
| - | <code bash> | + | === Networks === |
| - | $ docker stack deploy -c my_stack.yml lab5 | + | |
| - | </code> | + | |
| - | Once a service stack has been started, we can see its status by the following command: | + | The //**networks**// attribute describes the networks used in the configuration. Networks are passed in the form of objects. If we do not want to change the default configuration, the value is an empty field. An example of a network configuration is the following (where we use a network that already exists because it was created independently of the Docker Compose file): |
| - | <code bash> | + | <code yaml> |
| - | $ docker stack ps lab5 | + | networks: |
| - | + | my-network-that-actually-exists: | |
| - | ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS | + | external: true |
| - | cuktma92gm62 lab5_adminer.1 adminer:latest node2 Running Running 9 minutes ago | + | name: the-original-network-that-already-exists |
| - | njak2qzaobtt lab5_db.1 postgres:latest node1 Running Running 8 minutes ago | + | |
| - | m811buil7e63 lab5_backend.1 mobylab/backend:ia1 node1 Running Running 9 minutes ago | + | |
| - | jnfw37e34kz3 lab5_backend.2 mobylab/backend:ia1 node2 Running Running 9 minutes ago | + | |
| - | lkmy60wpy0gv lab5_visualizer.1 dockersamples/visualizer:latest node1 Running Running 9 minutes ago | + | |
| - | num87yijgxrg lab5_backend.3 mobylab/backend:ia1 node2 Running Running 9 minutes ago | + | |
| </code> | </code> | ||
| - | We can see the list of running service stacks using the following command: | + | In the example above, //**my-network-that-actually-exists**// is just a "renaming" of an existing network. |
| - | <code bash> | + | <note tip>The top-level property **//networks//** must be written at the same indentation level as **//services//** and it should not be confused with the child property **//networks//** within the service configuration.</note> |
| - | $ docker stack ls | + | |
| - | NAME SERVICES ORCHESTRATOR | + | ===== Docker Compose Commands ===== |
| - | lab5 4 Swarm | + | |
| - | </code> | + | |
| - | Furthermore, we can list all the services from all running stacks using the following command: | + | The commands for interacting with Docker Compose are syntactically similar to the classic Docker commands for both version 1 and version 2. Below, you can see the most used commands. For version 1, the utility is //**docker-compose**//, while for version 2 the utility is //**docker**// with the //**compose**// parameter. For more information, you can read [[https://docs.docker.com/compose/reference/|the official documentation for version 1]] and [[https://docs.docker.com/engine/reference/commandline/compose/|the official version 2 documentation]]. The main difference between versions 1 and 2 is that there are several additional commands in version 2. |
| <code bash> | <code bash> | ||
| - | $ docker service ls | + | $ docker-compose start # starts containers (V1) |
| + | $ docker-compose stop # stops containers (V1) | ||
| + | $ docker-compose pause # pauses containers using a SIGPAUSE signal (V1) | ||
| + | $ docker-compose unpause # removes containers from the pause state (V1) | ||
| + | $ docker-compose ps # lists active containers (V1) | ||
| + | $ docker-compose up # builds, recreates, starts and attaches containers to a service (V1) | ||
| + | $ docker-compose up -d # starts services in the background, detached from the terminal that initialised them (V1) | ||
| + | $ docker-compose up --build # creates images before starting containers (V1) | ||
| + | $ docker-compose -f my-compose.yml up # uses the specified Compose file instead of the default one (V1) | ||
| + | $ docker-compose down # stops containers and deletes them, along with the networks, volumes, and images created (V1) | ||
| + | $ docker-compose rm # deletes all stopped containers (V1) | ||
| + | $ docker-compose rm -s -v # with -s all containers are stopped and with -v the anonymous volumes attached are also deleted (V1) | ||
| - | ID NAME MODE REPLICAS IMAGE PORTS | + | $ docker compose start # starts containers (V2) |
| - | dekzzyais8g7 lab5_adminer replicated 1/1 adminer:latest *:8080->8080/tcp | + | $ docker compose stop # stops containers (V2) |
| - | ns9mxet1rkx5 lab5_db replicated 1/1 postgres:latest | + | $ docker compound pause # pauses containers using a SIGPAUSE signal (V2) |
| - | dh3sv3q74fy6 lab5_backend replicated 3/3 (max 2 per node) mobylab/backend:ia1 *:5555->80/tcp | + | $ docker compose unpause # removes containers from the pause state (V2) |
| - | ru0rd7g2ypu8 lab5_visualizer replicated 1/1 dockersamples/visualizer:latest *:8081->8080/tcp | + | $ docker compose ps # lists active containers (V2) |
| + | $ docker compose ls # lists all container stacks (V2) | ||
| + | $ docker compose -p my-proj -f my-comp.yml up # uses the specified Compose file instead of the default one and sets a project name (V2) | ||
| + | $ docker compose down # stops containers and deletes them, along with the networks, volumes, and images created (V2) | ||
| + | $ docker compose rm # deletes all stopped containers (V2) | ||
| + | $ docker compose rm -s -v # with -s all containers are stopped and with -v the anonymous volumes attached are also deleted (V2) | ||
| </code> | </code> | ||
| - | ===== Useful commands ===== | + | ===== Combining multiple Docker Compose files ===== |
| - | ==== Service interaction ==== | + | Docker Compose is recommended for use only at the local development stage. Docker Swarm (or other orchestrators, such as Kubernetes) should be used for testing or production environments. |
| - | <note tip>These commands can only be run on manager nodes.</note> | + | Precisely because it is used in the development stage, Docker Compose has a mechanism to combine several Compose files to create different running configurations without replicating the common parts. |
| - | <code bash> | + | To run a Compose configuration based on multiple YAML files, you can use the command below: |
| - | $ docker service create --name <SERVICE_NAME> <DOCKER_IMAGE> # creates a service based on an image | + | |
| - | $ docker service ls # lists all running services | + | |
| - | $ docker service inspect <SERVICE_NAME> # shows information about a service | + | |
| - | $ docker service logs –f <SERVICE_NAME> # shows a service's logs | + | |
| - | $ docker service ps <SERVICE_NAME> # shows a service's tasks and their statuses | + | |
| - | $ docker service update --replicas <N> <SERVICE_NAME> # updates a service by replicating its containers N times | + | |
| - | $ docker service rm <SERVICE_NAME> # removes a service | + | |
| - | </code> | + | |
| - | + | ||
| - | ==== Stack interaction ==== | + | |
| - | + | ||
| - | <note tip>These commands can only be run on manager nodes.</note> | + | |
| <code bash> | <code bash> | ||
| - | $ docker stack deploy -c <COMPOSE_FILE> <STACK_NAME> # creates a service stack based on a Compose file | + | $ docker-compose -f file-compose-1.yml -f file-compose-2.yml up --build # V1 |
| - | $ docker stack rm <STACK_NAME> # stops a service stack | + | $ docker compose -f file-compose-1.yml -f file-compose-2.yml up --build # V2 |
| - | $ docker stack ps <STACK_NAME> # lists the tasks of a running service stack | + | |
| - | $ docker stack ls # lists all running service stacks | + | |
| </code> | </code> | ||
| - | ==== Cluster interaction ==== | + | In the command above, the information in //**file-compose-2.yml**// will overwrite or complete the information in //**file-compose-1.yml**//. This is useful for quickly testing various configurations (or combinations of configurations). |
| - | + | ||
| - | <note tip>These commands can only be run on manager nodes.</note> | + | |
| - | + | ||
| - | <code bash> | + | |
| - | $ docker node ls # lists the nodes in the cluster | + | |
| - | $ docker node promote <NODE_NAME> # promotes a worker node to a manager | + | |
| - | $ docker node demote <NODE_NAME> # demotes a manager node to a worker | + | |
| - | $ docker swarm init [--advertise-addr <IP>] # creates a Docker Swarm cluster | + | |
| - | $ docker swarm join --token <TOKEN> <IP> # joins a Docker Swarm cluster | + | |
| - | </code> | + | |
| ====== Tasks ====== | ====== Tasks ====== | ||
| {{namespace>:ii:labs:s2:05:tasks&nofooter&noeditbutton}} | {{namespace>:ii:labs:s2:05:tasks&nofooter&noeditbutton}} | ||
| - | |||
| - | [[https://mobylabupb.com|{{:ii:labs:s2:moby_banner.png?direct&200|}}]] | ||
