crane: Cannot Append Windows Layer Without Duplicate "Files"

[ltsar-federated]

Describe the bug

When using crane append to append a TAR'd Windows layer to a base image, the resultant image fails crane validate with an error:
Error: validating layers: duplicate file path: Files

I understand that the Files path is a special entry for windows layers, but I'm not sure why this is occurring. There is no documentation anywhere about this issue but according to the docs for crane append this should "just work" on windows.

The layer I am appending is a built .NET Framework 4.8.1 MVC App onto the image mcr.microsoft.com/dotnet/framework/aspnet:4.8.1-windowsservercore-ltsc2022

To Reproduce

Use crane append to append a windows directory onto an existing windows image using a windows x64 host machine.

Expected behavior

I expected the files from the TAR'd layer to be appended successfully to the base image, and the resultant image be runnable.

Additional context

Running on a windows image builder for github actions.

• Output of crane version
0.19.1
• Registry used (e.g., GCR, ECR, Quay)
  ECR

[github-actions]

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Keep fresh with the 'lifecycle/frozen' label.

[ltsar-federated]

Maybe related to #1967 #1961

[ltsar-federated]

Confirmed still issue on 0.20.0

crane validate --tarball layer.tar
FAIL: layer.tar: validating layers: duplicate file path: Files
Error: validating layers: duplicate file path: Files

[ltsar-federated]

Maybe related to #1967 #1961

Probably not related. Going to read code again but this is really frustrating.

[ltsar-federated]

Failed to pull image "****/****:aaa1fd6": rpc error: code = Unknown desc = failed to pull and unpack image "****/****:aaa1fd6": failed to extract layer sha256:5c68f3291457ff8ac385f473f86a8f81abbfa041b164cc549c4d7369b015f270: mkdir \\?\C:\Windows\TEMP\hcs3793388198\Files: Cannot create a file when that file already exists.: unknown

not useful error message - weird, as Files is the directory root for the actual filesystem contents.

[ltsar-federated]

@jonjohnsonjr

Hello Jon,
I am wondering if you have any input on this issue. It's really escaping me as it seems like a bug in the implementation here but there's not a lot of love on the windows side. I am happy to contribute a patch if we can figure out what is causing this behavior.
I notice that the "Hives" subdirectory in the created docker image layer is empty. On a "working" version of this image, created with the docker daemon, that is filled with some dummy registry files.
Am I missing something here? Thank you in advance for your input.

The error is being thrown from this loop -

go-containerregistry/pkg/v1/validate/layer.go

Lines 141 to 153 in d36047a

	for {
	hdr, err := tarReader.Next()
	if errors.Is(err, io.EOF) {
	break
	}
	if err != nil {
	return nil, err
	}
	if _, ok := files[hdr.Name]; ok {
	return nil, fmt.Errorf("duplicate file path: %s", hdr.Name)
	}
	files[hdr.Name] = struct{}{}
	}

What I don't understand is why the "Files" subdir is being treated as an actual directory. It's supposed to be more-or-less ignored as I understand it (in Windows containers). Will continue digging.

[ltsar-federated]

When I examine the gzipped .tar layer that is not able to be decompressed, I see that the directory permissions for the Files and Hives headers are executable and readable. Every other layer in the image has no permissions for these files - they are just directories. I am wondering if this has something to do with it.

[ltsar-federated]

okay, I figured out what happened here.

The tar command when used with "." as a target broke the tar archive. using the parent directory name of the output files built a runnable container.