DAY19-Exploring Docker Volumes and Networks

we have learned how to create a docker-compose.yml file and push it to the Repository. Let's move forward and dive into advanced concepts of Docker Volume & Docker Network.

Docker Volume:

In Docker, a volume is a mechanism for persisting and sharing data between containers and the host machine. It provides a way to store and manage data separately from the container's lifecycle, allowing data to persist even if the container is stopped, restarted, or removed.

Here are some of the key features to understand about Docker volumes:

1. Data Persistence: Volumes enable persistent storage for containers. By using volumes, you can separate the data from the container's ephemeral storage, ensuring that data is preserved even when containers are recreated or destroyed.

2. Shared Storage: Volumes can be shared among multiple containers, allowing them to read from and write to the same data. This facilitates data sharing and collaboration between containers, making it useful for scenarios like database containers, shared file storage, or distributed systems.

3. Mount Points: Volumes are mounted into containers at specific mount points, which are directories within the container's filesystem. Any data written to these mount points is stored in the volume, and changes made to the volume are reflected across all containers that share it.

4. Volume Drivers: Docker supports different volume drivers that allow you to utilize external storage solutions or networked file systems as volumes. These drivers enable integration with platforms like Amazon S3, NFS, or cloud-based storage systems.

5. Volume Management: Docker provides commands and APIs for managing volumes. You can create, list, inspect, and remove volumes using Docker CLI or programmatically through Docker's API. This allows for easy management and manipulation of volumes.

6. Data Integrity and Consistency: Docker volumes provide mechanisms to ensure data integrity and consistency. When multiple containers write to the same volume, Docker handles concurrent access and ensures that changes made by one container are immediately visible to other containers.

Why Docker Volumes?

Docker volumes are the preferred way to save data over restarts of a Docker container. When compared to bind mounts, here are some of the advantages of volumes:

• Volumes are easier to back up, migrate and safer to share among containers

• Managing volumes can done from the Docker CLI or Docker API.

• Volume drivers allow volumes to be stored on remote hosts or cloud providers or to be encrypted.

• New volumes can be pre-populated by the container.

• Volumes do not increase the size of the container.

Docker has several types of volumes:

1. • Named Volumes: These are created and managed by Docker. You can specify a name for the volume, and Docker takes care of creating and managing the associated storage on the host machine.

     •        docker volume create somevolumename
              docker run -v name:/path/in/container ...
       

     • Host-mounted Volumes: With host-mounted volumes, you can directly mount a directory from the host machine into the container. This allows you to leverage the host machine's filesystem for storage.

     •        docker run -v /path/on/host:/path/in/container
       

       

     • Anonymous Volumes: Anonymous volumes are created and managed by Docker, but they are not explicitly given a name. They are typically used for temporary or disposable data.

     •        docker run -v /path/in/container ...
       

Some of the common docker volume commands:

🌀docker volume create : Creates a new named volume.

🌀docker volume ls Lists all the volumes that are currently available on the host machine.

🌀docker volume rm: Removes a named volume.

🌀docker volume inspect: Provides detailed information about a named volume.

🌀docker run -v: Mounts a host directory or file into a container.

🌀docker run --mount: Mounts a named volume or a host directory or file into a container.

Docker Networks:

The Docker network is a virtual network created by Docker to enable communication between Docker containers. If two containers are running on the same host they can communicate with each other without the need for ports to be exposed to the host machine.

For example, building an application that runs on a single Docker container will have a different network setup as compared to a web application with a cluster with database, application and load balancers which span multiple containers that need to communicate with each other. Additionally, clients from the outside world will need to access the web application container.

Docker Networking allows you to create a Network of Docker Containers managed by a master node called the manager. Containers inside the Docker Network can talk to each other by sharing packets of information.

Types of Docker Networks

Docker has specialized networking drivers. Docker network types:

• Bridge Network: The container's default. The default bridge network connects new containers without network specifications. Docker creates a host-private bridge network.

• As an example, consider you can have a Docker container running a web service on port 80. Because this container is attached to the bridge network on a private subnet, a port on the host system like 8000 needs to be mapped to port 80 on the container for outside traffic to reach the web service.

    $ docker network create -d bridge my-bridge-net
  

• Host Network: Docker hosts the network stack of a container using the host network mode. The container shares the host's network namespace and can directly connect to host-port services. No NAT translations provide the container with good network performance, but it loses network isolation.

• Overlay Network: The overlay network driver creates a distributed network over many Docker daemon servers, making it ideal for swarm services or containers that must communicate across multiple Docker hosts or clusters. Docker Swarm and multi-host systems require this.

    $ docker network create -d overlay --subnet=192.168.10.0/24 my-overlay-net
  

• Macvlan Network: Containers look like MAC-addressed devices in this network. It lets containers look like network devices to the rest of the network, which may help integrate them into existing networks. Containers have host-accessible IP addresses.

    $ docker network create -d macvlan \
      --subnet=192.168.40.0/24 \
      --gateway=192.168.40.1 \
      -o parent=eth0 my-macvlan-net
  

How Does Docker Networking Work?

• Docker File has the responsibility of building a Docker Image using the build command

• Docker Image contains all the project’s code.

• Using Docker Image, any user can run the code to create Docker Containers.

• Once Docker Image is built, it’s either uploaded to a registry or a Docker Hub

• Then, from Docker Hub, various teams such as Quality Assurance or Production teams will pull that image and prepare their own containers.

• These individual containers, communicate with each other through a network to perform the required actions, and this is nothing but Docker Networking.

• you can define Docker Networking as a communication passage through which all the isolated containers communicate with each other in various situations to perform the required actions

Now that you know how Docker networking works, it is important to understand the container network model.

Container Network Model

This concept will help you to build and deploy your applications in the Docker tool. Let’s discuss the components of the container network model in detail:

Network Sandbox

• It is an isolated sandbox that holds the network configuration of containers and is created when a user requests to generate an endpoint on the network.

  Endpoints

• It can have several endpoints in a network, as it represents a container’s network configuration such as IP address, MAC address, DNS, etc and establishes the connectivity for container services (within a network) with other services

• It helps in providing connectivity among the endpoints that belong to the same network and isolates them from the rest. So, whenever a network is created, or configuration is changed, the corresponding Network Driver will be notified with an event

  Docker Engine

• It is the base engine installed on your host machine to build and run containers using Docker components and services

• Its task is to manage the network with multiple drivers and provide the entry-point into libnetwork to maintain networks, whereas libnetwork supports multiple virtual drivers

  Network Drivers

• Docker supports networking for its containers via network drivers. These drivers have several network drivers.

• we will be discussing how to connect your containers with suitable network drivers. The network drivers used in Docker are below:

• Bridge

• Host

• None

• Overlay

• Macvlan

  Bridge

• It is a private default network created on the host and containers linked to this network have an internal IP address through which they communicate with each other easily.

  

• The Docker server (daemon) creates a virtual ethernet bridge docker0 that operates automatically, by delivering packets among various network interfaces

• These are widely used when applications are executed in a standalone container.

  Host

• It is a public network and utilizes the host’s IP address and TCP port space to display the services running inside the container

  

• It effectively disables network isolation between the docker host and the docker containers, which means using this network driver a user will be unable to run multiple containers on the same host

  None

• In this network driver, the Docker containers will neither have any access to external networks nor will it be able to communicate with other containers

• In simple terms, None is called a loopback interface, which means it has no external network interfaces. This option is used when a user wants to disable the networking access to a container.

  

  Overlay

• This is utilized for creating an internal private network to the Docker nodes in the Dockerswarm cluster and Kubernetes.

  

• It is an important network driver in Docker networking. It helps in providing the interaction between the stand-alone container and the Docker swarm service

  Macvlan

• It simplifies the communication process between containers

• This network assigns a MAC address to the Docker container. With this Mac address, the Docker server (daemon) routes the network traffic to a router

  

• It is suitable when a user wants to directly connect the container to the physical network rather than the Docker host.

  Benefits of Docker Network:

  1. Docker networks offer several benefits that make them a powerful tool for managing networking in containerized environments.

  1. This isolation helps prevent conflicts and interference between containers running on the same host.

  2. Docker networks facilitate efficient communication between containers in the same network using IP addresses. This simplifies networking configurations and improves performance.

  3. Network management simplifies tasks and allows for easy configuration and maintenance of networking components.

     5. Docker networks support load balancing and scaling of services. This improves application scalability, resiliency, and performance by distributing the workload among multiple instances of the same service.

     Commands used in Docker Networks:

     ✯Creates a new network: docker network create <networkname>

     ✺Removes a specified network: docker network rm <networkname>

     ✯Lists all networks: docker network ls

     ✺Connects a container to a network: docker network connect <networkname> <containername>

     ✯Displays detailed information about a network: docker network <inspect>

     ✺Disconnects a container from a network: docker network disconnect <networkname> <containername>

     Task-1

     Create a multi-container docker-compose file that will bring UP and bring DOWN containers in a single shot ( Example - Create application and database container )

     • Go to GitHub and copy the link of the app first, (https://github.com/chandanahdam/django-todo-cicd.git)

     git clone <repo link>
     git clone https://github.com/chandanahdam/django-todo-cicd.git
     

     Now, the repo is cloned into our local systems,

     • Create a new docker-compose.yml file and code below as follows,

     version : "3.3"
     services :
       my_web_app:
         container_name: "django-todo-app"
         build: .
         ports:
           - 8000:8000
         volumes:
           - django-todo-volume:/app
       my_db:
         container_name: "django-mysql-db"
         image: mysql:5.7
         ports:
           - 3306:3306
         environment:
           MYSQL_ROOT_PASSWORD: "test@123"
     volumes:
       django-todo-volume:
     

     • Use the docker-compose up command with the -d flag to start a multi-container application in detached mode.

       

     • Now check whether the container is running using docker-compose ps and multiple containers are now created using a single docker-compose file.

     • Check the logs using the docker-compose logs command.

       

     • Use docker-compose down command to stop and remove all containers, networks, and volumes associated with the application.

       

       Task-2

Learn how to use Docker Volumes and Named Volumes to share files and directories between multiple containers.

• Create volumes using the docker create volumes command.

• Using docker inspect <volume name>, see the information on the created volumes

  

• Create two or more containers that share the same named volume using the docker run --mount command.

    #using --mount creating 2 containers
    sudo docker run -d -p 8000:8000 --mount source=my_volume,target=/app django-todo-app:latest
    sudo docker run -d -p 8001:8001 --mount source=my_volume,target=/app django-todo-cicd_my_web_app:latest
  

• The docker run --mount and docker run -v commands serve similar purposes in that they both allow you to mount volumes or bind mounts into a Docker container, but they have different syntax and provide different levels of flexibility and here -v means a shorter and simpler way to specify volume mounts.

  

• Now check inside the containers whether the data is the same in all the containers using the command docker exec -it <container name>sh

  

• Use the docker volume ls command to list all volumes

• The docker volume rm command to remove the volume when you're done.

  

we’ve covered the what and how of Docker networking in detail, starting with Docker’s network drivers available out-of-the-box and We ran through some examples of the most common Docker network and volume commands. This provides you with a decent overview of how Docker networks and volumes.

Thank you, for giving your precious time to reading the blog. Give it a like and share your suggestions it will be helpful!!!!!!!!