Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose copy.Options.ForceCompressionFormat #2343

Open
stephenjudkins opened this issue May 29, 2024 · 6 comments
Open

Expose copy.Options.ForceCompressionFormat #2343

stephenjudkins opened this issue May 29, 2024 · 6 comments
Labels
kind/feature A request for, or a PR adding, new functionality stale-issue

Comments

@stephenjudkins
Copy link

Steps to reproduce:

  • Generate a local docker archive (test.tar) that uses uncompressed tarballs as layers
  • Run skopeo copy docker-archive:test.tar dir://out
  • Observe that the manifest in out/manifest.json has layers specified as "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip"
  • Observe that the actual layer files are uncompressed tarballs: e719c9bac8ba3f1aac6ab6d3dbd006be2bee3dca4030e0fca96297ba72bdb569/layer.tar: POSIX tar archive
@mtrmac
Copy link
Contributor

mtrmac commented May 30, 2024

Thanks for your report. This is tracked in containers/image#1270 ; fixing that could break uploads to some registries.

As described there, the code now allows forcing compression to happen, ~avoiding the MIME type inconsistency. That option does not currently exist in Skopeo, it could be added.

@mtrmac mtrmac changed the title skopeo copy with uncompressed layers input creates manifest incorrectly reporting layers are compressed. docker-archive creates manifest incorrectly reporting layers are compressed. May 30, 2024
@stephenjudkins
Copy link
Author

Got it. My workaround now is to first push to a dir: and compress and then push the repository, which isn't too burdensome. Would be nice if skopeo could do this directly. I'll also look at a PR in the upstream tool (nix dockertools) that's creating the upstream package and seeing if it can just compress the layers there

@stephenjudkins
Copy link
Author

To be clear, this was not docker-archive. It was Nix dockerTools. Here's the manifest.json it creates:

[
    {
        "Config": "13644826e3861fa657c4f2f92507dd1cfa49df418e415848c1ba8236a0b9ec7d.json",
        "RepoTags": [
            "lambdaify-cowsay:sd7qj2zb884f1vr04vbvqkg7pi21lkjb"
        ],
        "Layers": [
            "e719c9bac8ba3f1aac6ab6d3dbd006be2bee3dca4030e0fca96297ba72bdb569/layer.tar",
            "bc66d165bafbd6e4f8ae0b7b4f7e2c2ee7ed104ec033064ae2eb2c5fb8b9ba13/layer.tar",
            "ffdc9f7eef9b8ab8eaa574fbb46b6bae67da05e28a06673a050e65e282fc8075/layer.tar",
            "e3a28d402ebe81d6a75162bd44f6fb9c8cc36573d7442d1424e3935ab05cf658/layer.tar",
            "d15644b70db264dbb5e3276d8eb4270e38f4e4948979b6d4a2e9e0dc7e4b9a9b/layer.tar",
            "209087dc5746df140db7b23044954a231e757f9cd8d47ae72c573e79cc51e8b9/layer.tar",
            "767b335f27daae8a8eb46a23237b7e53317a888389451617c51d328dd4522893/layer.tar",
            "ff09b2f841e0a24bd19d286a920b152e2bfb69e78474f9d2b098f3cc9ca3651d/layer.tar",
            "237ac55de4e0cd14fc0a46899bb6a0d57dcbffd669ce06c3d9d8b2b3b66eb2ea/layer.tar",
            "3117be9a0a806e63f62c269522673770113a6700fe3be7b59889bdf5fc0b22d4/layer.tar",
            "47f7e9765eb2f923ad057b94a467e68bd88527fa1c1760264b8b8d2179be72fc/layer.tar",
            "12e851f11e10e943c8f0255bbe1b38e033389c3d1a1ffcda2be329636aca02c9/layer.tar",
            "b9322a4589b2492fbc9a3b6836e4f9d4b81a5a9b290a5d91c78daaa138b56200/layer.tar",
            "d2a303717fe9d19f091f594b8cdbbbf3260b94869582b56cf4136aacdd69e25d/layer.tar",
            "12166ccb22c63af4103ee487b560c2d30c71659509beeb02cf7003a03400eafa/layer.tar",
            "da381933e68eca7e83ff1cd35fe906b4c4e5d13310aeea8c0669914e89281b57/layer.tar",
            "af8eccd692f27910da2222754a3782c0939b3adc936d7813c0805d044b18ddd1/layer.tar",
            "eb2e6e1da446e0f69a3beedf3b23e7c61b310617957526c4636c798263dfc8b2/layer.tar"
        ]
    }
]

@mtrmac
Copy link
Contributor

mtrmac commented May 30, 2024

Would you care to file a Skopeo PR to expose containers/image#2068 ? That might be the simplest way to do this in one step.

(“docker-archive creates” refers to the in-memory manifest created when reading the docker save-formatted tarballs.)

@stephenjudkins
Copy link
Author

Thanks for the context!

@mtrmac mtrmac changed the title docker-archive creates manifest incorrectly reporting layers are compressed. Expose copy.Options.ForceCompressionFormat Jun 27, 2024
@mtrmac mtrmac added the kind/feature A request for, or a PR adding, new functionality label Jun 27, 2024
Copy link

A friendly reminder that this issue had no activity for 30 days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature A request for, or a PR adding, new functionality stale-issue
Projects
None yet
Development

No branches or pull requests

2 participants