Low ulimits when building with docker-compose/podman-compose

[jRilkef]

Issue Description

When an image is build with docker-compose or podman-compose, the nofile ulimit is 1024 which is too low.

Steps to reproduce the issue

Build the following with either podman-compose or docker-compose that's using podman socket:

# Containerfile
FROM docker.io/library/alpine:3
RUN cat /proc/self/limits

# docker-compose.yml
version: "3.8"
services:
  test:
    image: localhost/test:1
    build:
      dockerfile: Containerfile
      context: .

Describe the results you received

...
STEP 2/2: RUN cat /proc/self/limits
Limit                     Soft Limit           Hard Limit           Units     
Max open files            1024                 1024                 files     
...

Describe the results you expected

I expect this limit to be configurable. Podman has the following overrides set up:

# /etc/security/limits.d/100-podman.conf
...
root soft nofile 131072
root hard nofile 524288
* soft nofile 131072
* hard nofile 524288

# /etc/containers/containers.conf AND /usr/share/containers/containers.conf 
[containers]
default_ulimits = ["nofile=131072:524288",]
...

However, builds using *-compose are not honoring those overrides.

podman info output

$ podman info

host:
  arch: amd64
  buildahVersion: 1.33.2
  cgroupControllers:
  - cpu
  - io
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.8-2.fc39.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.8, commit: '
  cpuUtilization:
    idlePercent: 98.78
    systemPercent: 0.87
    userPercent: 0.35
  cpus: 4
  databaseBackend: sqlite
  distribution:
    distribution: fedora
    variant: coreos
    version: "39"
  eventLogger: journald
  freeLocks: 2048
  hostname: localhost.localdomain
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
    uidmap:
    - container_id: 0
      host_id: 501
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
  kernel: 6.6.8-200.fc39.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 627404800
  memTotal: 2059902976
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: aardvark-dns-1.9.0-1.fc39.x86_64
      path: /usr/libexec/podman/aardvark-dns
      version: aardvark-dns 1.9.0
    package: netavark-1.9.0-1.fc39.x86_64
    path: /usr/libexec/podman/netavark
    version: netavark 1.9.0
  ociRuntime:
    name: crun
    package: crun-1.12-1.fc39.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.12
      commit: ce429cb2e277d001c2179df1ac66a470f00802ae
      rundir: /run/user/501/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +LIBKRUN +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: passt-0^20231204.gb86afe3-1.fc39.x86_64
    version: |
      pasta 0^20231204.gb86afe3-1.fc39.x86_64
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/user/501/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.2-1.fc39.x86_64
    version: |-
      slirp4netns version 1.2.2
      commit: 0ee2d87523e906518d34a6b423271e4826f71faf
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 0h 18m 19.00s
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphRootAllocated: 106769133568
  graphRootUsed: 3016437760
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 2
  runRoot: /run/user/501/containers
  transientStore: false
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 4.8.2
  Built: 1702300984
  BuiltTime: Mon Dec 11 15:23:04 2023
  GitCommit: ""
  GoVersion: go1.21.4
  Os: linux
  OsArch: linux/amd64
  Version: 4.8.2

Podman in a container

No

Privileged Or Rootless

None

Upstream Latest Release

Yes

Additional environment details

I'm facing the same issue with podman on macOS and with privileged podman in a container.

Additional information

n/a

[rhatdan]

$ ulimit -n
1024

Podman and no other process in rootless mode can override the limits set for the user running the podman command.

If you raise the limits for the user, then the ulimit should increase.

[jRilkef]

During build time, both soft and hard limits seem to inherit from the user's soft limit. During run time, however (aka podman run --entrypoint cat alpine /proc/self/limits), both inherit from the user's hard limit. Is that expected behavior?

[mheon]

That sounds like a bug - Buildah grabbing different limits from Podman. Question would be which to standardize on; I would vote for hard limit as that's likely higher, and we see a lot of folks running out of resources.

[Miaourt]

In the meantime that the fix get downstreamed on every podman installs, is there a solution to manually up the limit beside running as root ?