Looks like the host compiler is being used for some tests

[rossburton]

When comparing a supposedly identical cross build on two different build hosts the output is slightly different. A diffoscope analysis of the build has this fragment in:

│ │ ├── ./usr/lib/perl5/5.30.1/x86_64-linux/CORE/config.h
│ │ │ @@ -2317,15 +2317,15 @@
│ │ │  #define HAS_BUILTIN_ADD_OVERFLOW	/**/
│ │ │  #define HAS_BUILTIN_SUB_OVERFLOW	/**/
│ │ │  #define HAS_BUILTIN_MUL_OVERFLOW	/**/
│ │ │  
│ │ │  /* HAS_C99_VARIADIC_MACROS:
│ │ │   *	If defined, the compiler supports C99 variadic macros.
│ │ │   */
│ │ │ -#define	HAS_C99_VARIADIC_MACROS	/**/
│ │ │ +/*#define	HAS_C99_VARIADIC_MACROS	/ **/
│ │ │  
│ │ │  /* HAS_CLASS:
│ │ │   *	This symbol, if defined, indicates that the class routine is
│ │ │   *	available to classify doubles.  Available for example in AIX.
│ │ │   *	The returned values are defined in <float.h> and are:
│ │ │   *
│ │ │   *	FP_PLUS_NORM	Positive normalized, nonzero
│ │ ├── ./usr/lib/perl5/5.30.1/x86_64-linux/CORE/xconfig.h
│ │ │ @@ -2217,17 +2217,17 @@
│ │ │   *	Note that this variable is initialized from the sig_num_init,
│ │ │   *	not from sig_num (which is unused).
│ │ │   */
│ │ │  /* SIG_SIZE:
│ │ │   *	This variable contains the number of elements of the SIG_NAME
│ │ │   *	and SIG_NUM arrays, excluding the final NULL entry.
│ │ │   */
│ │ │ -#define SIG_NAME "ZERO", "HUP", "INT", "QUIT", "ILL", "TRAP", "ABRT", "BUS", "FPE", "KILL", "USR1", "SEGV", "USR2", "PIPE", "ALRM", "TERM", "STKFLT", "CHLD", "CONT", "STOP", "TSTP", "TTIN", "TTOU", "URG", "XCPU", "XFSZ", "VTALRM", "PROF", "WINCH", "IO", "PWR", "SYS", "IOT", "CLD", "POLL", "UNUSED", 0		/**/
│ │ │ -#define SIG_NUM  0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 6, 17, 29, 31, 0		/**/
│ │ │ -#define SIG_SIZE 36			/**/
│ │ │ +#define SIG_NAME "ZERO", "HUP", "INT", "QUIT", "ILL", "TRAP", "ABRT", "BUS", "FPE", "KILL", "USR1", "SEGV", "USR2", "PIPE", "ALRM", "TERM", "STKFLT", "CHLD", "CONT", "STOP", "TSTP", "TTIN", "TTOU", "URG", "XCPU", "XFSZ", "VTALRM", "PROF", "WINCH", "IO", "PWR", "SYS", "IOT", "CLD", "POLL", 0	/**/
│ │ │ +#define SIG_NUM  0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 6, 17, 29, 0		/**/
│ │ │ +#define SIG_SIZE 35			/**/
│ │ │  
│ │ │  /* USE_CROSS_COMPILE:
│ │ │   *	This symbol, if defined, indicates that Perl is being cross-compiled.
│ │ │   */
│ │ │  /* PERL_TARGETARCH:
│ │ │   *	This symbol, if defined, indicates the target architecture
│ │ │   *	Perl has been cross-compiled to.  Undefined if not a cross-compile.
│ │ │ @@ -2317,15 +2317,15 @@
│ │ │  #define HAS_BUILTIN_ADD_OVERFLOW	/**/
│ │ │  #define HAS_BUILTIN_SUB_OVERFLOW	/**/
│ │ │  #define HAS_BUILTIN_MUL_OVERFLOW	/**/
│ │ │  
│ │ │  /* HAS_C99_VARIADIC_MACROS:
│ │ │   *	If defined, the compiler supports C99 variadic macros.
│ │ │   */
│ │ │ -#define	HAS_C99_VARIADIC_MACROS	/**/
│ │ │ +/*#define	HAS_C99_VARIADIC_MACROS	/ **/
│ │ │  
│ │ │  /* HAS_CLASS:
│ │ │   *	This symbol, if defined, indicates that the class routine is
│ │ │   *	available to classify doubles.  Available for example in AIX.
│ │ │   *	The returned values are defined in <float.h> and are:
│ │ │   *
│ │ │   *	FP_PLUS_NORM	Positive normalized, nonzero
│ │ │ @@ -3079,15 +3079,15 @@
│ │ │   *	include <xlocale.h> to get uselocale() and its friends.
│ │ │   */
│ │ │  #define	HAS_NEWLOCALE	/**/
│ │ │  #define	HAS_FREELOCALE	/**/
│ │ │  #define	HAS_USELOCALE	/**/
│ │ │  #define	HAS_DUPLOCALE	/**/
│ │ │  /*#define	HAS_QUERYLOCALE	/ **/
│ │ │ -#define	I_XLOCALE               /**/
│ │ │ +/*#define	I_XLOCALE               / **/
│ │ │  
│ │ │  /* HAS_NEXTAFTER:
│ │ │   *	This symbol, if defined, indicates that the nextafter routine is
│ │ │   *	available to return the next machine representable double from
│ │ │   *	x in direction y.
│ │ │   */
│ │ │  #define HAS_NEXTAFTER		/**/
│ │ │ @@ -3692,15 +3692,15 @@
│ │ │   */
│ │ │  /*#define	I_PROT		/ **/
│ │ │  
│ │ │  /* I_QUADMATH:
│ │ │   *	This symbol, if defined, indicates that <quadmath.h> exists and
│ │ │   *	should be included.
│ │ │   */
│ │ │ -/*#define	I_QUADMATH		/ **/
│ │ │ +#define	I_QUADMATH		/**/
│ │ │  
│ │ │  /* I_SHADOW:

The hunch is that the host compiler is being used when probing, instead of the cross. Do you have any thoughts or ideas?

[arsv]

Quick check to verify that: open config.log, and search for "variadic". There will be two check in case of a cross build, the first one for host-miniperl and the second one for the target perl executable.

Checking C99 variadic macros
| #include <stdio.h>
| #define foo(fmt, ...) printf(fmt, __VA_ARGS__)
| int main(void) { foo("%i\n", 1234); return 0; }
> aarch64-linux-gnu-gcc -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -c -o try.o try.c
Setting d_c99_variadic_macros=define
Result: supported

As far as I can tell, it uses the target compiler and otherwise works like it's supposed to. However, it's a good idea to check anyway, in case there are some unexpected issues there. Maybe even try diffing config.log-s from the two hosts, assuming the same configuration there should be very little difference there normally.

You can also copy the test code from the log and run the command manually.

[rpurdie]

I think this is a dash vs bash as /bin/sh issue. If I take the "\n" out the test, in config.log I see:
Checking C99 variadic macros
| #include <stdio.h>
| #define foo(fmt, ...) printf(fmt, VA_ARGS)
| int main(void) { foo("%i", 1234); return 0; }

gcc -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -c -o try.o try.c
Setting d_c99_variadic_macros=define
Result: supported

but with "\n" in, I see:
Checking C99 variadic macros
| #include <stdio.h>
| #define foo(fmt, ...) printf(fmt, VA_ARGS)
| int main(void) { foo("%i
| ", 1234); return 0; }

gcc -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -c -o try.o try.c
try.c: In function ‘main’:
try.c:3:22: warning: missing terminating " character
int main(void) { foo("%i
^
try.c:4:1: warning: missing terminating " character
--

try.c:3:18: error: ‘foo’ undeclared (first use in this function); did you mean ‘feof’?
int main(void) { foo("%i
^~~
feof
try.c:3:18: note: each undeclared identifier is reported only once for each function it appears in
try.c:3:1: error: expected ‘;’ at end of input
int main(void) { foo("%i
^~~
try.c:3:1: error: expected declaration or statement at end of input
Setting d_c99_variadic_macros=undef
Result: missing

If I then change /bin/sh to point at bash, it works again.

The question is therefore whether the scripts should require bash, whether the "\n" should be removed and/or whether there are other issues?

[arsv]

Ah, I see. This one is easy to fix, there's no need to have \n there at all, it's completely irrelevant for the test. I'll try running the rest with dash, the other diff chunks probably have similar causes.

Current versions are supposed to work with non-bash shells. The project did originally use #!/bin/bash but got switched to #!/bin/sh later, with code changes to avoid relying on bash-specific functionality.

[rpurdie]

We've done a lot more tests and whilst the bash/dash issue is definitely the cause of the variadic test change, we cannot isolate the other changes or reproduce them in an environment we can debug. It doesn't seem to be related to the bash/dash issue. The only thought we have right now is some kind of leakage of host config into the target config but how/where this happening, we don't know. Could they race against each other due to parallelism somehow on a loaded system?

[kanavin]

I think we package configuration files for miniperl (native helper executable that is used in building target perl), which are specific for the host compiler and vary from one host to the next (xconfig.h, xconfig.sh and maybe more). I'm looking into fixing it.

[rossburton]

I think we package configuration files for miniperl (native helper executable that is used in building target perl), which are specific for the host compiler and vary from one host to the next (xconfig.h, xconfig.sh and maybe more). I'm looking into fixing it.

That would explain why the package changes, yes. If xconfig.sh is specific to the host then we can definitely just not ship that and the bulk of the changes disappear (apart from the variadic one, which is fixed), right? Surely nothing needs xconfig.sh when installed.

Alex, you deserve a medal for noticing the obvious.

[rpurdie]

These differences are in config.h, not xconfig.h though so whilst that could be a problem, I don't think its the issue we're seeing?

[rpurdie]

Sorry, I clearly misread that output, the other issues are clearly host based, thanks for pointing out what was staring me in the face! :)

[kanavin]

Patch:
http://lists.openembedded.org/pipermail/openembedded-core/2020-February/292501.html

[rpurdie]

There is one further piece of fallout from the bash vs. dash issue which is the setting of trml. With one it is "\n" with the other it becomes "".

[arsv]

Removed all \n uses. That trnl value isn't even used anywhere.

And yeah xconfig.h is a host-specific file that should not be packaged.

[rossburton]

And yeah xconfig.h is a host-specific file that should not be packaged.

Note that we're not installing it explicitly, make install is doing that.

[kanavin]

make install is basically installing *.h, so maybe it's best to delete the file after the fact like my patch does. Also note that some headers, perl.h in particular actually include xconfig.h (subject to some define which I didn't investigate), so this may break builds on target that involve that header.

[arsv]

Since it happens in make install, then yeah deleting it after the fact is probably the best option. The alternative is patching installperl.

perl.h includes xconfig.h in place of config.h when -DUSE_CROSS_COMPILE is passed to the compiler, which only normally happens when building miniperl. That's how they switch between perl and mini-perl configurations. Should not be a problem unless somebody explicitly defines that somewhere.