Docker Volumes Data: 8 Easy Tests and Mistakes to Avoid

Docker volumes data persistence looks simple until a database container loses its files, a non-root user cannot write, or a prune command deletes a volume you wanted. The old version of this post explained the storage architecture in theory. This rewrite tests it: Docker 29.9.2 running in a privileged container, with the overlayfs storage driver and the root directory /var/lib/docker. Every output below is from that run. Two parts of the old post could not be tested here and are marked as such at the end: NFS or cloud volume drivers, and SELinux labels.

Test 1: the container layer is not Docker volumes data

$ docker run --name t1 alpine:3.22 sh -c "echo data > /file; cat /file"
data
$ docker rm t1
$ docker run --rm alpine:3.22 ls /file
ls: /file: No such file or directory

Whatever a container writes outside a volume or mount lives in its writable layer and disappears with the container. That is the reason for Docker volumes data in the first place. The official Docker volumes documentation describes the same model.

Test 2: named Docker volumes data survives and is shared

$ docker volume create pgdata
$ docker volume inspect pgdata
        "Driver": "local",
        "Mountpoint": "/var/lib/docker/volumes/pgdata/_data",
        "Name": "pgdata",
        "Scope": "local"
$ docker run --rm -v pgdata:/var/lib/app alpine:3.22 sh -c "echo from-first > /var/lib/app/f"
$ docker run --rm -v pgdata:/var/lib/app alpine:3.22 cat /var/lib/app/f
from-first

A second container read what the first one wrote, and the files sit under the Mountpoint on the Docker host. Docker protects a volume in use:

$ docker volume rm pgdata
Error response from daemon: remove pgdata: volume is in use - [e9445c87b6846223c31c624e5c7e5b96c537b21c8f1667c098c07a5f27374666]

After the container is removed, docker volume rm pgdata succeeded.

Test 3: anonymous Docker volumes data stays behind

A path with no name, such as -v /data, creates a volume with a 64-character ID as its name. Removing the container does not remove it:

$ docker run -d --name a1 -v /data alpine:3.22 sleep 300
$ docker rm -f a1
$ docker volume ls
2e019ae9cdb036cd31289950af0ac95f4847d74d1ae8b7ee4050c408716cb57c anonymous=yes
named1 anonymous=no

(The anonymous= label is added by my format string, which checks for a 64-character name.) Use docker rm -v to delete the container together with its anonymous volumes: with docker rm -f -v anon2 the volume count dropped to 0. Without -v, forgotten anonymous volumes pile up. This is the quiet disk leak behind many "no space left" incidents.

Test 4: first-use copy of Docker volumes data, nocopy and bind mounts

I built an image with /app/data/seed.txt inside it. A new named volume mounted over that path is filled with the image content on first use. The nocopy option skips that, and a bind mount hides the image content:

$ docker run --rm -v newvol:/app/data seeded ls /app/data
seed.txt
$ docker run --rm -v bindtest:/app/data:nocopy seeded ls /app/data
(empty)
$ docker run --rm -v /hostdir:/app/data seeded ls /app/data
(empty: the host directory hides the image content)

This is why an application that ships default config in a data directory works with a named volume and breaks with a bind mount of an empty host folder.

Docker volumes data tests: permission denied for a non-root user, bind mount error, tmpfs full and prune behaviour
Four errors and behaviours from the Docker 29.9.2 test.

Test 5: permissions for Docker volumes data and a non-root user

$ docker run --rm --user 1000:1000 -v permvol:/data alpine:3.22 sh -c "touch /data/x && echo ok"
touch: /data/x: Permission denied
$ docker run --rm -v permvol:/data alpine:3.22 ls -ld /data
drwxr-xr-x    1 root     root             0 Nov  2 08:29 /data

A new empty volume holding Docker volumes data is owned by root, so a container running as user 1000 cannot write to it. Two fixes worked. The first changes the owner once: docker run --rm -v permvol:/data alpine:3.22 chown 1000:1000 /data, after which the touch printed ok. The second builds the right owner into the image, with mkdir /data && chown 1000:1000 /data and USER 1000:1000; a new volume mounted there was copied with the image's ownership (drwxr-xr-x 1000 1000 /data) and the write worked. Remember the first-use copy from Test 4: the second fix only works for volumes that are empty when first mounted.

Test 6: bind mounts next to Docker volumes data

$ docker run --rm -v /nope/path:/d alpine:3.22 ls -ld /d
drwxr-xr-x    1 root     root             0 Nov  2 08:29 /d
$ ls -d /nope/path
/nope/path
$ docker run --rm --mount type=bind,src=/nope2,dst=/d alpine:3.22 true
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /nope2

With -v, which is the short form many people use for Docker volumes data too, a typo in the host path silently creates an empty directory and the container starts with no data. With --mount you get an error, which is safer. A read-only mount (:ro) is enforced:

$ docker run --rm -v permvol:/data:ro alpine:3.22 sh -c "touch /data/y"
touch: /data/y: Read-only file system

Test 7: tmpfs and size-limited Docker volumes data

$ docker run --rm --tmpfs /scratch:size=10m alpine:3.22 sh -c "dd if=/dev/zero of=/scratch/big bs=1M count=20"
dd: error writing '/scratch/big': No space left on device
10+0 records out
$ docker volume create --driver local --opt type=tmpfs --opt device=tmpfs --opt o=size=20m ramvol
$ docker run --rm -v ramvol:/r alpine:3.22 df -h /r | tail -1
tmpfs                    20.0M         0     20.0M   0% /r

Data on tmpfs lives in memory, so it is the one kind of Docker volumes data you do not keep. and is gone when the container stops, so use it only for scratch data, with a size limit. The local driver accepts mount options; I tested the tmpfs type only. NFS uses the same --opt mechanism, but I had no NFS server, so I did not test it.

Test 8: back up and restore Docker volumes data with tar

$ docker run --rm -v permvol:/data:ro -v /backup:/backup alpine:3.22 tar czf /backup/permvol.tgz -C /data .
$ docker volume create restored
$ docker run --rm -v restored:/data -v /backup:/backup:ro alpine:3.22 tar xzf /backup/permvol.tgz -C /data
$ docker run --rm -v restored:/data alpine:3.22 cat /data/keep.txt
precious

Mount the source read-only for the backup, and restore into a new volume. For databases, stop the container or use the database's own dump tool first; a tar of live database files was not tested. For the same reason, the Redis data directory in the Redis on Rocky Linux post needs a proper save before you copy it.

Test 9: prune and Docker volumes data you want to keep

$ docker volume prune -f
Deleted Volumes:
2e019ae9cdb036cd31289950af0ac95f4847d74d1ae8b7ee4050c408716cb57c
$ docker volume ls --format "{{.Name}}"
named1
$ docker volume prune -a -f      # named1 was in use by a running container
Deleted Volumes:
named2
$ docker rm -f u1; docker volume prune -a -f
Deleted Volumes:
named1

The prune result is the most surprising part of this Docker volumes data test. On Docker 29.9.2, plain docker volume prune deleted only the anonymous volume. Named volumes were kept until I added -a, and even then a volume used by a running container survived. Once the container was removed, prune -a deleted named1, data included. Never run prune -a on a host where a stopped container holds data you still want.

Test 10: compose down and Docker volumes data

$ docker compose down
 Container c-app-1 Removed
$ docker volume ls --format "{{.Name}}"
c_appdata
$ docker compose up -d; docker compose exec app cat /data/f
hi
$ docker compose down -v
 Volume c_appdata Removed

The data written before docker compose down was still there after up. Only down -v removed the volume. Compose prefixes the volume with the project name (c_appdata from the folder name c), which matters when you back up by name. The image build side of Docker 29.9.2 is covered in the BuildKit post.

Docker volumes data mistakes to avoid

  • Using an anonymous volume and removing the container without -v.
  • Mounting a new volume for a non-root user without fixing ownership.
  • Typing a bind path with -v, which creates an empty directory instead of failing.
  • Running docker volume prune -a while stopped containers hold data.
  • Using docker compose down -v out of habit.
  • Backing up a live database by copying files.
  • Treating the writable layer as if it were Docker volumes data.

What this test did not cover

The old post also discussed NFS and cloud block storage drivers, and the SELinux labels :z and :Z. The test host used neither, so I cannot show output for them. Check the official documentation for those, and test them on a staging host first.

FAQ

Where is Docker volumes data stored on Linux? Under the Mountpoint shown by docker volume inspect, here /var/lib/docker/volumes/pgdata/_data.

Does docker rm delete my Docker volumes data? Not for named volumes. Anonymous volumes survive plain docker rm and are deleted with -v.

How do I move a volume to another host? Use the tar pattern in Test 8: make the archive on the old host, copy it, and restore it into a new volume.

Comments

Popular posts from this blog

How to Play Minecraft Bedrock Edition on Linux: A Comprehensive Guide for Tech Professionals

Linux desktop frozen? My 5 go-to tricks to try - before forcing a hard reboot

The Ultimate Guide: How to Set Up DXVK in Wine on Linux for Enhanced Gaming Performance