Differences between local symbols emitted into binary by SwiftC/LLVM on Windows & Linux vs. MacOS?

Hi, I'm having an issue with a pull request I've written, and am not sure where to go from here.

My pull request uses swift::lookupSymbol in order to find out the names of keypath getters. On MacOS, and presumably iOS, this works fine in the scenarios I've tested. However, on Windows and Linux, it fails.

My first step in investigating this was to try to inspect the output of a typical invocation of swiftc. Compiling this code on iOS:

open class S {
    
    open var a: Int = 7
    
}

let a = \S.a

and then running nm on the output gives this result:

0000000100003bfc T _$s14dladdr_subject1SC1aSivM
0000000100003c3c t _$s14dladdr_subject1SC1aSivM.resume.0
0000000100003f24 S _$s14dladdr_subject1SC1aSivMTq
0000000100003b6c T _$s14dladdr_subject1SC1aSivg
0000000100003f14 S _$s14dladdr_subject1SC1aSivgTq
0000000100003d3c t _$s14dladdr_subject1SC1aSivpACTK
0000000100003d9c t _$s14dladdr_subject1SC1aSivpACTk
0000000100003eb0 S _$s14dladdr_subject1SC1aSivpMV
0000000100003eb8 S _$s14dladdr_subject1SC1aSivpWvd
0000000100003b60 T _$s14dladdr_subject1SC1aSivpfi
0000000100003bb4 T _$s14dladdr_subject1SC1aSivs
0000000100003f1c S _$s14dladdr_subject1SC1aSivsTq
0000000100003cd8 T _$s14dladdr_subject1SCACycfC
0000000100003f2c s _$s14dladdr_subject1SCACycfCTq
0000000100003d10 t _$s14dladdr_subject1SCACycfc
0000000100003f5c s _$s14dladdr_subject1SCMF
0000000100003df8 T _$s14dladdr_subject1SCMa
00000001000080e0 d _$s14dladdr_subject1SCMf
00000001000080b8 D _$s14dladdr_subject1SCMm
0000000100003ee0 S _$s14dladdr_subject1SCMn
00000001000080f0 D _$s14dladdr_subject1SCN
0000000100003c9c T _$s14dladdr_subject1SCfD
0000000100003c78 T _$s14dladdr_subject1SCfd
0000000100008168 s _$s14dladdr_subject1as24ReferenceWritableKeyPathCyAA1SCSiGvp
0000000100003ed0 s _$s14dladdr_subjectMXM
                 U _$sBoWV
                 U _OBJC_CLASS_$__TtCs12_SwiftObject
                 U _OBJC_METACLASS_$__TtCs12_SwiftObject
0000000100008070 s __DATA__TtC14dladdr_subject1S
0000000100008048 s __IVARS__TtC14dladdr_subject1S
0000000100008000 s __METACLASS_DATA__TtC14dladdr_subject1S
0000000100003f34 s ___swift_reflection_version
0000000100000000 T __mh_execute_header
                 U __objc_empty_cache
0000000100003b38 T _main
                 U _objc_opt_self
                 U _swift_allocObject
                 U _swift_beginAccess
                 U _swift_deallocClassInstance
                 U _swift_endAccess
                 U _swift_getKeyPath
                 U _swift_release
                 U _swift_retain
0000000100003e82 s _symbolic Si
0000000100003e7c s _symbolic _____ 14dladdr_subject1SC

As you can see, nearly every symbol here is a mangled Swift name, even the local ones. So it's not a surprise that dladdr works.

On Windows, on the other hand, the output of running objdump /All on the lib file output by SwiftC only contains mangled names for the global symbols.

	1	File Type: LIBRARY
	2	
	3	 
	4	Archive member name at 8: /
	5	
	6	FFFFFFFF time/date
	7	
	8	         uid
	9	
	10	         gid
	11	
	12	       0 mode
	13	
	14	     2B4 size
	15	
	16	correct header end
	17	
	18	 
	19	    28 public symbols
	20	
	21	 
	22	      600 __IMPORT_DESCRIPTOR_test2
	23	
	24	      81A __NULL_IMPORT_DESCRIPTOR
	25	
	26	      94A ⌂test2_NULL_THUNK_DATA
	27	
	28	     10AE __imp_main
	29	
	30	     10AE main
	31	
	32	      D16 $s5test21SC1aSivpfi
	33	
	34	      D16 __imp_$s5test21SC1aSivpfi
	35	
	36	      B6E $s5test21SC1aSivg
	37	
	38	      B6E __imp_$s5test21SC1aSivg
	39	
	40	      D80 $s5test21SC1aSivs
	41	
	42	      D80 __imp_$s5test21SC1aSivs
	43	
	44	      A9C $s5test21SC1aSivM
	45	
	46	      A9C __imp_$s5test21SC1aSivM
	47	
	48	     104A $s5test21SCfd
	49	
	50	     104A __imp_$s5test21SCfd
	51	
	52	      FE6 $s5test21SCfD
	53	
	54	      FE6 __imp_$s5test21SCfD
	55	
	56	      E52 $s5test21SCACycfC
	57	
	58	      E52 __imp_$s5test21SCACycfC
	59	
	60	      EBA $s5test21SCMa
	61	
	62	      EBA __imp_$s5test21SCMa
	63	
	64	      C40 __imp_$s5test21SC1aSivpMV
	65	
	66	      CAA __imp_$s5test21SC1aSivpWvd
	67	
	68	      F1E __imp_$s5test21SCMn
	69	
	70	      BD6 __imp_$s5test21SC1aSivgTq
	71	
	72	      DE8 __imp_$s5test21SC1aSivsTq
	73	
	74	      B04 __imp_$s5test21SC1aSivMTq
	75	
	76	      F82 __imp_$s5test21SCN
	77	
	78	 
	79	Archive member name at 2F8: /
	80	
	81	FFFFFFFF time/date
	82	
	83	         uid
	84	
	85	         gid
	86	
	87	       0 mode
	88	
	89	     2CC size
	90	
	91	correct header end
	92	
	93	 
	94	    19 offsets
	95	
	96	 
	97	        1      600
	98	
	99	        2      81A
	100	
	101	        3      94A
	102	
	103	        4     10AE
	104	
	105	        5      D16
	106	
	107	        6      B6E
	108	
	109	        7      D80
	110	
	111	        8      A9C
	112	
	113	        9     104A
	114	
	115	        A      FE6
	116	
	117	        B      E52
	118	
	119	        C      EBA
	120	
	121	        D      C40
	122	
	123	        E      CAA
	124	
	125	        F      F1E
	126	
	127	       10      BD6
	128	
	129	       11      DE8
	130	
	131	       12      B04
	132	
	133	       13      F82
	134	
	135	 
	136	    28 public symbols
	137	
	138	 
	139	        8 $s5test21SC1aSivM
	140	
	141	        6 $s5test21SC1aSivg
	142	
	143	        5 $s5test21SC1aSivpfi
	144	
	145	        7 $s5test21SC1aSivs
	146	
	147	        B $s5test21SCACycfC
	148	
	149	        C $s5test21SCMa
	150	
	151	        A $s5test21SCfD
	152	
	153	        9 $s5test21SCfd
	154	
	155	        1 __IMPORT_DESCRIPTOR_test2
	156	
	157	        2 __NULL_IMPORT_DESCRIPTOR
	158	
	159	        8 __imp_$s5test21SC1aSivM
	160	
	161	       12 __imp_$s5test21SC1aSivMTq
	162	
	163	        6 __imp_$s5test21SC1aSivg
	164	
	165	       10 __imp_$s5test21SC1aSivgTq
	166	
	167	        D __imp_$s5test21SC1aSivpMV
	168	
	169	        E __imp_$s5test21SC1aSivpWvd
	170	
	171	        5 __imp_$s5test21SC1aSivpfi
	172	
	173	        7 __imp_$s5test21SC1aSivs
	174	
	175	       11 __imp_$s5test21SC1aSivsTq
	176	
	177	        B __imp_$s5test21SCACycfC
	178	
	179	        C __imp_$s5test21SCMa
	180	
	181	        F __imp_$s5test21SCMn
	182	
	183	       13 __imp_$s5test21SCN
	184	
	185	        A __imp_$s5test21SCfD
	186	
	187	        9 __imp_$s5test21SCfd
	188	
	189	        4 __imp_main
	190	
	191	        4 main
	192	
	193	        3 ⌂test2_NULL_THUNK_DATA
	194	
	195	 
	196	Archive member name at 600: test2/
	197	
	198	FFFFFFFF time/date
	199	
	200	         uid
	201	
	202	         gid
	203	
	204	       0 mode
	205	
	206	     1DD size
	207	
	208	correct header end
	209	
	210	 
	211	FILE HEADER VALUES
	212	
	213	            8664 machine (x64)
	214	
	215	               3 number of sections
	216	
	217	        D4AF237A time date stamp
	218	
	219	              FF file pointer to symbol table
	220	
	221	               8 number of symbols
	222	
	223	               0 size of optional header
	224	
	225	               0 characteristics
	226	
	227	 
	228	SECTION HEADER #1
	229	
	230	.debug$S name
	231	
	232	       0 physical address
	233	
	234	       0 virtual address
	235	
	236	      3B size of raw data
	237	
	238	      8C file pointer to raw data (0000008C to 000000C6)
	239	
	240	       0 file pointer to relocation table
	241	
	242	       0 file pointer to line numbers
	243	
	244	       0 number of relocations
	245	
	246	       0 number of line numbers
	247	
	248	42100040 flags
	249	
	250	         Initialized Data
	251	
	252	         Discardable
	253	
	254	         1 byte align
	255	
	256	         Read Only
	257	
	258	 
	259	RAW DATA #1
	260	
	261	  00000000: 02 00 00 00 0C 00 09 00 00 00 00 00 05 74 65 73  .............tes
	262	
	263	  00000010: 74 32 27 00 13 10 07 00 00 00 D0 00 00 00 00 00  t2'.......Ð.....
	264	
	265	  00000020: 00 00 0E 00 19 00 C4 6F 12 4D 69 63 72 6F 73 6F  ......Äo.Microso
	266	
	267	  00000030: 66 74 20 28 52 29 20 4C 49 4E 4B                 ft (R) LINK
	268	
	269	 
	270	SECTION HEADER #2
	271	
	272	.idata$2 name
	273	
	274	       0 physical address
	275	
	276	       0 virtual address
	277	
	278	      14 size of raw data
	279	
	280	      C7 file pointer to raw data (000000C7 to 000000DA)
	281	
	282	      DB file pointer to relocation table
	283	
	284	       0 file pointer to line numbers
	285	
	286	       3 number of relocations
	287	
	288	       0 number of line numbers
	289	
	290	C0300040 flags
	291	
	292	         Initialized Data
	293	
	294	         4 byte align
	295	
	296	         Read Write
	297	
	298	 
	299	RAW DATA #2
	300	
	301	  00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
	302	
	303	  00000010: 00 00 00 00                                      ....
	304	
	305	 
	306	RELOCATIONS #2
	307	
	308	                                                Symbol    Symbol
	309	
	310	 Offset    Type              Applied To         Index     Name
	311	
	312	 --------  ----------------  -----------------  --------  ------
	313	
	314	 0000000C  ADDR32NB                   00000000         3  .idata$6
	315	
	316	 00000000  ADDR32NB                   00000000         4  .idata$4
	317	
	318	 00000010  ADDR32NB                   00000000         5  .idata$5
	319	
	320	 
	321	SECTION HEADER #3
	322	
	323	.idata$6 name
	324	
	325	       0 physical address
	326	
	327	       0 virtual address
	328	
	329	       6 size of raw data
	330	
	331	      F9 file pointer to raw data (000000F9 to 000000FE)
	332	
	333	      DB file pointer to relocation table
	334	
	335	       0 file pointer to line numbers
	336	
	337	       0 number of relocations
	338	
	339	       0 number of line numbers
	340	
	341	C0200040 flags
	342	
	343	         Initialized Data
	344	
	345	         2 byte align
	346	
	347	         Read Write
	348	
	349	 
	350	RAW DATA #3
	351	
	352	  00000000: 74 65 73 74 32 00                                test2.
	353	
	354	 
	355	COFF SYMBOL TABLE
	356	
	357	000 01016FC4 ABS    notype       Static       | @comp.id
	358	
	359	001 00000000 SECT2  notype       External     | __IMPORT_DESCRIPTOR_test2
	360	
	361	002 C0000040 SECT2  notype       Section      | .idata$2
	362	
	363	003 00000000 SECT3  notype       Static       | .idata$6
	364	
	365	004 C0000040 UNDEF  notype       Section      | .idata$4
	366	
	367	005 C0000040 UNDEF  notype       Section      | .idata$5
	368	
	369	006 00000000 UNDEF  notype       External     | __NULL_IMPORT_DESCRIPTOR
	370	
	371	007 00000000 UNDEF  notype       External     | ⌂test2_NULL_THUNK_DATA
	372	
	373	 
	374	String Table Size = 0x4E bytes
	375	
	376	 
	377	Archive member name at 81A: test2/
	378	
	379	FFFFFFFF time/date
	380	
	381	         uid
	382	
	383	         gid
	384	
	385	       0 mode
	386	
	387	      F4 size
	388	
	389	correct header end
	390	
	391	 
	392	FILE HEADER VALUES
	393	
	394	            8664 machine (x64)
	395	
	396	               2 number of sections
	397	
	398	        E46365C2 time date stamp
	399	
	400	              B3 file pointer to symbol table
	401	
	402	               2 number of symbols
	403	
	404	               0 size of optional header
	405	
	406	               0 characteristics
	407	
	408	 
	409	SECTION HEADER #1
	410	
	411	.debug$S name
	412	
	413	       0 physical address
	414	
	415	       0 virtual address
	416	
	417	      3B size of raw data
	418	
	419	      64 file pointer to raw data (00000064 to 0000009E)
	420	
	421	       0 file pointer to relocation table
	422	
	423	       0 file pointer to line numbers
	424	
	425	       0 number of relocations
	426	
	427	       0 number of line numbers
	428	
	429	42100040 flags
	430	
	431	         Initialized Data
	432	
	433	         Discardable
	434	
	435	         1 byte align
	436	
	437	         Read Only
	438	
	439	 
	440	RAW DATA #1
	441	
	442	  00000000: 02 00 00 00 0C 00 09 00 00 00 00 00 05 74 65 73  .............tes
	443	
	444	  00000010: 74 32 27 00 13 10 07 00 00 00 D0 00 00 00 00 00  t2'.......Ð.....
	445	
	446	  00000020: 00 00 0E 00 19 00 C4 6F 12 4D 69 63 72 6F 73 6F  ......Äo.Microso
	447	
	448	  00000030: 66 74 20 28 52 29 20 4C 49 4E 4B                 ft (R) LINK
	449	
	450	 
	451	SECTION HEADER #2
	452	
	453	.idata$3 name
	454	
	455	       0 physical address
	456	
	457	       0 virtual address
	458	
	459	      14 size of raw data
	460	
	461	      9F file pointer to raw data (0000009F to 000000B2)
	462	
	463	       0 file pointer to relocation table
	464	
	465	       0 file pointer to line numbers
	466	
	467	       0 number of relocations
	468	
	469	       0 number of line numbers
	470	
	471	C0300040 flags
	472	
	473	         Initialized Data
	474	
	475	         4 byte align
	476	
	477	         Read Write
	478	
	479	 
	480	RAW DATA #2
	481	
	482	  00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
	483	
	484	  00000010: 00 00 00 00                                      ....
	485	
	486	 
	487	COFF SYMBOL TABLE
	488	
	489	000 01016FC4 ABS    notype       Static       | @comp.id
	490	
	491	001 00000000 SECT2  notype       External     | __NULL_IMPORT_DESCRIPTOR
	492	
	493	 
	494	String Table Size = 0x1D bytes
	495	
	496	 
	497	Archive member name at 94A: test2/
	498	
	499	FFFFFFFF time/date
	500	
	501	         uid
	502	
	503	         gid
	504	
	505	       0 mode
	506	
	507	     116 size
	508	
	509	correct header end
	510	
	511	 
	512	FILE HEADER VALUES
	513	
	514	            8664 machine (x64)
	515	
	516	               3 number of sections
	517	
	518	        C594922D time date stamp
	519	
	520	              D7 file pointer to symbol table
	521	
	522	               2 number of symbols
	523	
	524	               0 size of optional header
	525	
	526	               0 characteristics
	527	
	528	 
	529	SECTION HEADER #1
	530	
	531	.debug$S name
	532	
	533	       0 physical address
	534	
	535	       0 virtual address
	536	
	537	      3B size of raw data
	538	
	539	      8C file pointer to raw data (0000008C to 000000C6)
	540	
	541	       0 file pointer to relocation table
	542	
	543	       0 file pointer to line numbers
	544	
	545	       0 number of relocations
	546	
	547	       0 number of line numbers
	548	
	549	42100040 flags
	550	
	551	         Initialized Data
	552	
	553	         Discardable
	554	
	555	         1 byte align
	556	
	557	         Read Only
	558	
	559	 
	560	RAW DATA #1
	561	
	562	  00000000: 02 00 00 00 0C 00 09 00 00 00 00 00 05 74 65 73  .............tes
	563	
	564	  00000010: 74 32 27 00 13 10 07 00 00 00 D0 00 00 00 00 00  t2'.......Ð.....
	565	
	566	  00000020: 00 00 0E 00 19 00 C4 6F 12 4D 69 63 72 6F 73 6F  ......Äo.Microso
	567	
	568	  00000030: 66 74 20 28 52 29 20 4C 49 4E 4B                 ft (R) LINK
	569	
	570	 
	571	SECTION HEADER #2
	572	
	573	.idata$5 name
	574	
	575	       0 physical address
	576	
	577	       0 virtual address
	578	
	579	       8 size of raw data
	580	
	581	      C7 file pointer to raw data (000000C7 to 000000CE)
	582	
	583	       0 file pointer to relocation table
	584	
	585	       0 file pointer to line numbers
	586	
	587	       0 number of relocations
	588	
	589	       0 number of line numbers
	590	
	591	C0400040 flags
	592	
	593	         Initialized Data
	594	
	595	         8 byte align
	596	
	597	         Read Write
	598	
	599	 
	600	RAW DATA #2
	601	
	602	  00000000: 00 00 00 00 00 00 00 00                          ........
	603	
	604	 
	605	SECTION HEADER #3
	606	
	607	.idata$4 name
	608	
	609	       0 physical address
	610	
	611	       0 virtual address
	612	
	613	       8 size of raw data
	614	
	615	      CF file pointer to raw data (000000CF to 000000D6)
	616	
	617	       0 file pointer to relocation table
	618	
	619	       0 file pointer to line numbers
	620	
	621	       0 number of relocations
	622	
	623	       0 number of line numbers
	624	
	625	C0400040 flags
	626	
	627	         Initialized Data
	628	
	629	         8 byte align
	630	
	631	         Read Write
	632	
	633	 
	634	RAW DATA #3
	635	
	636	  00000000: 00 00 00 00 00 00 00 00                          ........
	637	
	638	 
	639	COFF SYMBOL TABLE
	640	
	641	000 01016FC4 ABS    notype       Static       | @comp.id
	642	
	643	001 00000000 SECT2  notype       External     | ⌂test2_NULL_THUNK_DATA
	644	
	645	 
	646	String Table Size = 0x1B bytes
	647	
	648	 
	649	Archive member name at A9C: test2/
	650	
	651	FFFFFFFF time/date
	652	
	653	         uid
	654	
	655	         gid
	656	
	657	       0 mode
	658	
	659	      2C size
	660	
	661	correct header end
	662	
	663	 
	664	  Version      : 0
	665	
	666	  Machine      : 8664 (x64)
	667	
	668	  TimeDateStamp: EF21AC44
	669	
	670	  SizeOfData   : 00000018
	671	
	672	  DLL name     : test2
	673	
	674	  Symbol name  : $s5test21SC1aSivM
	675	
	676	  Type         : code
	677	
	678	  Name type    : name
	679	
	680	  Hint         : 0
	681	
	682	  Name         : $s5test21SC1aSivM
	683	
	684	 
	685	Archive member name at B04: test2/
	686	
	687	FFFFFFFF time/date
	688	
	689	         uid
	690	
	691	         gid
	692	
	693	       0 mode
	694	
	695	      2E size
	696	
	697	correct header end
	698	
	699	 
	700	  Version      : 0
	701	
	702	  Machine      : 8664 (x64)
	703	
	704	  TimeDateStamp: ED8B155D
	705	
	706	  SizeOfData   : 0000001A
	707	
	708	  DLL name     : test2
	709	
	710	  Symbol name  : $s5test21SC1aSivMTq
	711	
	712	  Type         : data
	713	
	714	  Name type    : name
	715	
	716	  Hint         : 1
	717	
	718	  Name         : $s5test21SC1aSivMTq
	719	
	720	 
	721	Archive member name at B6E: test2/
	722	
	723	FFFFFFFF time/date
	724	
	725	         uid
	726	
	727	         gid
	728	
	729	       0 mode
	730	
	731	      2C size
	732	
	733	correct header end
	734	
	735	 
	736	  Version      : 0
	737	
	738	  Machine      : 8664 (x64)
	739	
	740	  TimeDateStamp: E17E2F46
	741	
	742	  SizeOfData   : 00000018
	743	
	744	  DLL name     : test2
	745	
	746	  Symbol name  : $s5test21SC1aSivg
	747	
	748	  Type         : code
	749	
	750	  Name type    : name
	751	
	752	  Hint         : 2
	753	
	754	  Name         : $s5test21SC1aSivg
	755	
	756	 
	757	Archive member name at BD6: test2/
	758	
	759	FFFFFFFF time/date
	760	
	761	         uid
	762	
	763	         gid
	764	
	765	       0 mode
	766	
	767	      2E size
	768	
	769	correct header end
	770	
	771	 
	772	  Version      : 0
	773	
	774	  Machine      : 8664 (x64)
	775	
	776	  TimeDateStamp: FFBB266C
	777	
	778	  SizeOfData   : 0000001A
	779	
	780	  DLL name     : test2
	781	
	782	  Symbol name  : $s5test21SC1aSivgTq
	783	
	784	  Type         : data
	785	
	786	  Name type    : name
	787	
	788	  Hint         : 3
	789	
	790	  Name         : $s5test21SC1aSivgTq
	791	
	792	 
	793	Archive member name at C40: test2/
	794	
	795	FFFFFFFF time/date
	796	
	797	         uid
	798	
	799	         gid
	800	
	801	       0 mode
	802	
	803	      2E size
	804	
	805	correct header end
	806	
	807	 
	808	  Version      : 0
	809	
	810	  Machine      : 8664 (x64)
	811	
	812	  TimeDateStamp: D00EED8D
	813	
	814	  SizeOfData   : 0000001A
	815	
	816	  DLL name     : test2
	817	
	818	  Symbol name  : $s5test21SC1aSivpMV
	819	
	820	  Type         : data
	821	
	822	  Name type    : name
	823	
	824	  Hint         : 4
	825	
	826	  Name         : $s5test21SC1aSivpMV
	827	
	828	 
	829	Archive member name at CAA: test2/
	830	
	831	FFFFFFFF time/date
	832	
	833	         uid
	834	
	835	         gid
	836	
	837	       0 mode
	838	
	839	      2F size
	840	
	841	correct header end
	842	
	843	 
	844	  Version      : 0
	845	
	846	  Machine      : 8664 (x64)
	847	
	848	  TimeDateStamp: C11B660F
	849	
	850	  SizeOfData   : 0000001B
	851	
	852	  DLL name     : test2
	853	
	854	  Symbol name  : $s5test21SC1aSivpWvd
	855	
	856	  Type         : data
	857	
	858	  Name type    : name
	859	
	860	  Hint         : 5
	861	
	862	  Name         : $s5test21SC1aSivpWvd
	863	
	864	 
	865	Archive member name at D16: test2/
	866	
	867	FFFFFFFF time/date
	868	
	869	         uid
	870	
	871	         gid
	872	
	873	       0 mode
	874	
	875	      2E size
	876	
	877	correct header end
	878	
	879	 
	880	  Version      : 0
	881	
	882	  Machine      : 8664 (x64)
	883	
	884	  TimeDateStamp: F06F5B23
	885	
	886	  SizeOfData   : 0000001A
	887	
	888	  DLL name     : test2
	889	
	890	  Symbol name  : $s5test21SC1aSivpfi
	891	
	892	  Type         : code
	893	
	894	  Name type    : name
	895	
	896	  Hint         : 6
	897	
	898	  Name         : $s5test21SC1aSivpfi
	899	
	900	 
	901	Archive member name at D80: test2/
	902	
	903	FFFFFFFF time/date
	904	
	905	         uid
	906	
	907	         gid
	908	
	909	       0 mode
	910	
	911	      2C size
	912	
	913	correct header end
	914	
	915	 
	916	  Version      : 0
	917	
	918	  Machine      : 8664 (x64)
	919	
	920	  TimeDateStamp: E946CB15
	921	
	922	  SizeOfData   : 00000018
	923	
	924	  DLL name     : test2
	925	
	926	  Symbol name  : $s5test21SC1aSivs
	927	
	928	  Type         : code
	929	
	930	  Name type    : name
	931	
	932	  Hint         : 7
	933	
	934	  Name         : $s5test21SC1aSivs
	935	
	936	 
	937	Archive member name at DE8: test2/
	938	
	939	FFFFFFFF time/date
	940	
	941	         uid
	942	
	943	         gid
	944	
	945	       0 mode
	946	
	947	      2E size
	948	
	949	correct header end
	950	
	951	 
	952	  Version      : 0
	953	
	954	  Machine      : 8664 (x64)
	955	
	956	  TimeDateStamp: DE2E3E0F
	957	
	958	  SizeOfData   : 0000001A
	959	
	960	  DLL name     : test2
	961	
	962	  Symbol name  : $s5test21SC1aSivsTq
	963	
	964	  Type         : data
	965	
	966	  Name type    : name
	967	
	968	  Hint         : 8
	969	
	970	  Name         : $s5test21SC1aSivsTq
	971	
	972	 
	973	Archive member name at E52: test2/
	974	
	975	FFFFFFFF time/date
	976	
	977	         uid
	978	
	979	         gid
	980	
	981	       0 mode
	982	
	983	      2C size
	984	
	985	correct header end
	986	
	987	 
	988	  Version      : 0
	989	
	990	  Machine      : 8664 (x64)
	991	
	992	  TimeDateStamp: C9CDC753
	993	
	994	  SizeOfData   : 00000018
	995	
	996	  DLL name     : test2
	997	
	998	  Symbol name  : $s5test21SCACycfC
	999	
	1000	  Type         : code
	1001	
	1002	  Name type    : name
	1003	
	1004	  Hint         : 9
	1005	
	1006	  Name         : $s5test21SCACycfC
	1007	
	1008	 
	1009	Archive member name at EBA: test2/
	1010	
	1011	FFFFFFFF time/date
	1012	
	1013	         uid
	1014	
	1015	         gid
	1016	
	1017	       0 mode
	1018	
	1019	      28 size
	1020	
	1021	correct header end
	1022	
	1023	 
	1024	  Version      : 0
	1025	
	1026	  Machine      : 8664 (x64)
	1027	
	1028	  TimeDateStamp: DDBE0AB4
	1029	
	1030	  SizeOfData   : 00000014
	1031	
	1032	  DLL name     : test2
	1033	
	1034	  Symbol name  : $s5test21SCMa
	1035	
	1036	  Type         : code
	1037	
	1038	  Name type    : name
	1039	
	1040	  Hint         : 10
	1041	
	1042	  Name         : $s5test21SCMa
	1043	
	1044	 
	1045	Archive member name at F1E: test2/
	1046	
	1047	FFFFFFFF time/date
	1048	
	1049	         uid
	1050	
	1051	         gid
	1052	
	1053	       0 mode
	1054	
	1055	      28 size
	1056	
	1057	correct header end
	1058	
	1059	 
	1060	  Version      : 0
	1061	
	1062	  Machine      : 8664 (x64)
	1063	
	1064	  TimeDateStamp: C2664BA4
	1065	
	1066	  SizeOfData   : 00000014
	1067	
	1068	  DLL name     : test2
	1069	
	1070	  Symbol name  : $s5test21SCMn
	1071	
	1072	  Type         : data
	1073	
	1074	  Name type    : name
	1075	
	1076	  Hint         : 11
	1077	
	1078	  Name         : $s5test21SCMn
	1079	
	1080	 
	1081	Archive member name at F82: test2/
	1082	
	1083	FFFFFFFF time/date
	1084	
	1085	         uid
	1086	
	1087	         gid
	1088	
	1089	       0 mode
	1090	
	1091	      27 size
	1092	
	1093	correct header end
	1094	
	1095	 
	1096	  Version      : 0
	1097	
	1098	  Machine      : 8664 (x64)
	1099	
	1100	  TimeDateStamp: F98C3218
	1101	
	1102	  SizeOfData   : 00000013
	1103	
	1104	  DLL name     : test2
	1105	
	1106	  Symbol name  : $s5test21SCN
	1107	
	1108	  Type         : data
	1109	
	1110	  Name type    : name
	1111	
	1112	  Hint         : 12
	1113	
	1114	  Name         : $s5test21SCN
	1115	
	1116	 
	1117	Archive member name at FE6: test2/
	1118	
	1119	FFFFFFFF time/date
	1120	
	1121	         uid
	1122	
	1123	         gid
	1124	
	1125	       0 mode
	1126	
	1127	      28 size
	1128	
	1129	correct header end
	1130	
	1131	 
	1132	  Version      : 0
	1133	
	1134	  Machine      : 8664 (x64)
	1135	
	1136	  TimeDateStamp: F6FD66FF
	1137	
	1138	  SizeOfData   : 00000014
	1139	
	1140	  DLL name     : test2
	1141	
	1142	  Symbol name  : $s5test21SCfD
	1143	
	1144	  Type         : code
	1145	
	1146	  Name type    : name
	1147	
	1148	  Hint         : 13
	1149	
	1150	  Name         : $s5test21SCfD
	1151	
	1152	 
	1153	Archive member name at 104A: test2/
	1154	
	1155	FFFFFFFF time/date
	1156	
	1157	         uid
	1158	
	1159	         gid
	1160	
	1161	       0 mode
	1162	
	1163	      28 size
	1164	
	1165	correct header end
	1166	
	1167	 
	1168	  Version      : 0
	1169	
	1170	  Machine      : 8664 (x64)
	1171	
	1172	  TimeDateStamp: C3E4DFEA
	1173	
	1174	  SizeOfData   : 00000014
	1175	
	1176	  DLL name     : test2
	1177	
	1178	  Symbol name  : $s5test21SCfd
	1179	
	1180	  Type         : code
	1181	
	1182	  Name type    : name
	1183	
	1184	  Hint         : 14
	1185	
	1186	  Name         : $s5test21SCfd
	1187	
	1188	 
	1189	Archive member name at 10AE: test2/
	1190	
	1191	FFFFFFFF time/date
	1192	
	1193	         uid
	1194	
	1195	         gid
	1196	
	1197	       0 mode
	1198	
	1199	      1F size
	1200	
	1201	correct header end
	1202	
	1203	 
	1204	  Version      : 0
	1205	
	1206	  Machine      : 8664 (x64)
	1207	
	1208	  TimeDateStamp: EA7BFC31
	1209	
	1210	  SizeOfData   : 0000000B
	1211	
	1212	  DLL name     : test2
	1213	
	1214	  Symbol name  : main
	1215	
	1216	  Type         : code
	1217	
	1218	  Name type    : name
	1219	
	1220	  Hint         : 15
	1221	
	1222	  Name         : main
	1223	
	1224	 
	1225	     Exports
	1226	
	1227	 
	1228	       ordinal    name
	1229	
	1230	 
	1231	                  $s5test21SC1aSivM
	1232	
	1233	                  $s5test21SC1aSivMTq
	1234	
	1235	                  $s5test21SC1aSivg
	1236	
	1237	                  $s5test21SC1aSivgTq
	1238	
	1239	                  $s5test21SC1aSivpMV
	1240	
	1241	                  $s5test21SC1aSivpWvd
	1242	
	1243	                  $s5test21SC1aSivpfi
	1244	
	1245	                  $s5test21SC1aSivs
	1246	
	1247	                  $s5test21SC1aSivsTq
	1248	
	1249	                  $s5test21SCACycfC
	1250	
	1251	                  $s5test21SCMa
	1252	
	1253	                  $s5test21SCMn
	1254	
	1255	                  $s5test21SCN
	1256	
	1257	                  $s5test21SCfD
	1258	
	1259	                  $s5test21SCfd
	1260	
	1261	                  main
	1262	
	1263	 
	1264	  Summary
	1265	
	1266	 
	1267	          B1 .debug$S
	1268	
	1269	          14 .idata$2
	1270	
	1271	          14 .idata$3
	1272	
	1273	           8 .idata$4
	1274	
	1275	           8 .idata$5
	1276	
	1277	           6 .idata$6

I have yet to check Linux (I don't have a Linux machine or VM available), but I assume it will be a similar story there, too.

So now I'm not sure where to go from here. Does this count as a bug in Swift? In LLVM? Or is the behavior for non-global symbols an implementation detail of these platforms?

cc @compnerd

Are you sure that the .lib is a static library? If it is the import library it will only contain the public interfaces that are meant to be usable from an external module. The import descriptor and null import descriptor make me believe that you are looking at an import library.

Secondly, the lookup of a non-public symbol is problematic. The symbol table is not guaranteed to be present, and thus the lookup by name may fail. If a symbol is not public and does not participate in dynamic linking, it may be fully internalised and discarded. If you need to recover the symbol at runtime, it would be advisable to generate a LUT, possibly with a perfect hash under WMO. This applies to both ELFish and COFFish platforms.

Are you sure that the .lib is a static library?

It's the output of running swiftc on a single Swift file (not named main.swift); I don't know whether the output of that should be a static library or not, but I assume it would be. I have the .lib file and a "exports library file" as the two outputs of invoking swiftc.

To be honest I am not at all knowledgable about both Windows dev or what the ramifications of this being static vs. dynamic would be, especially in the context of the local symbols.

the lookup of a non-public symbol is problematic. The symbol table is not guaranteed to be present, and thus the lookup by name may fail

The goal of the proposal is to emit info for debugging use only, on a best effort basis. So it's okay if this doesn't always work, as long as it works most of the time in debug builds. I realize now I probably should have explained in the original post, apologies for not doing so.

If you need to recover the symbol at runtime, it would be advisable to generate a LUT, possibly with a perfect hash under WMO

I would definitely use a lookup table if we wanted to generate metadata that was reliably useful for a more serious, production usage, but I'm not sure if that would be acceptable for something like implementing debugDescription. Certainly we wouldn't want to output it in release builds for code size reasons.

@Joe_Groff just curious how you feel about lookup tables for KeyPath names/stable identifiers in general? Opt-in Encodable KeyPaths would definitely need them, maybe in future we would want to make the default be to emit them for all keypaths in debug builds?

This applies to both ELFish and COFFish platforms.

Is there something I can do to make these symbols show up somewhat reliably in debug builds on those platforms?

If the answer to the above is "no," how do you feel about this not being supported on Windows (and maybe getting support later if the appropriate lookup table is added for some other purpose)?

It depends on the product type. If it is an executable or library, it should be an import library. Building a static library on Windows is currently only possible manually or with CMake or bazel not with SPM.

You can convert them to public ABI. That should enforce that the symbols participate in dynamic linking and thus are guaranteed. This will of course also increase load times as well. Except perhaps in the main binary :thinking: ... unless we build with -rdynamic for ELFish targets.

Emitting some additional metadata for opt-in codable key paths is definitely a must, and it would make sense for debug aides like your CustomDebugStringConvertible implementation to take advantage of it when available.

Thanks for explaining. So based on this, my tentative plan is modify getOrCreateKeyPathGetter() in SILGenExpr.cpp to output public symbols if the following is true:

isDebug && (target.isOSWindows() || target.isOSLinux())

I ran into two issues trying to do that last weekend:

  1. Figuring out how to determine if we're making a debug build. I found a flag related to determining whether assertions should crash or not which is probably a good indication, but I don't think I should reuse it. Maybe I should go off optimization levels and emit these when the library is built with -ONone?
  2. When I tested always making the getters public on MacOS (I figured if it worked on MacOS I could push it and it would have a good chance of passing CI on Linux/Windows), the compiler crashed because the symbols weren't present in the TBD file. Example output:
<unknown>:0: error: symbol '$ss7Float16V13SIMD64StorageVyABSicipADTK' (key path getter for Swift.Float16.SIMD64Storage.subscript(Swift.Int) -> Swift.Float16 : Swift.Float16.SIMD64Storage) is in generated IR file, but not in TBD file

My understanding is that TBD files are a MacOS/iOS only thing, but I'm guessing the TBD checks are still done on Windows/Linux, and maybe it just emits a different file format? If that's the case then I guess I'll need to figure out how to modify TBDGen.cpp to add these to the TBD file, or maybe turn off the checks in the TBD verifier. The former seems preferable.

Does this seem reasonable?

Use of -Onone is the correct thing to key the check off of. That is what differentiates a debug vs non-debug build.

Correct, you will need to adjust the TBD emission. The TBD emission is ensuring that the emitted symbol list matches the declarations. Fixing that should give you a working implementation for the other platforms as well. However, the compiler should not crash, though it should report the error that the TBD is inconsistent. The TBD is the moral equivalent to the import library on Windows and the .dynsym segment in the ELF binary (as there is no separation of the link and run time components). Disabling the TBDGen checks seem like a bad idea to me as these checks ensure that the ABI boundary remains precise. It also gives you a good way to test the behaviour of the compiler as you change the semantics of the ABI for the functionality.

You should not make symbols public only for the sake of debuggability. That will also impede dead code elimination and other optimizations that we could do assuming that the symbols are not used externally. This seems like something that should be addressed more holistically than for key paths—is there a way we could make DWARF/PDB info available to the runtime for symbol lookup when present?

1 Like

With PDB, it is an external source, that may be entirely unavailable.

The thing is, with DWARF, it is possible to use in binary debug info for debug builds (at the cost of ~10x memory usage, ~2x CPU usage, and increased build times). We would also need to impede the ability to use strip on the binary as one may strip the binary dropping the necessary data, which I'm not sure how to accomplish. We should also ensure that we filter out support for fission should we go down that route.

Practically speaking, I think that the solution is that we simply always emit the lookup table. This would ensure that it is safe and stable and doesn't come at an extraordinary cost for debug builds.

The goal of using lookupSymbol here is only to provide a best effort, though, and shouldn't be a barrier to stripping symbols or doing other optimizations in release builds. The symbols in question aren't guaranteed to be present on any platform, including Apple ones. We will eventually emit lookup metadata for properties that opt in to key path codability, but I don't think it's a good idea to have that lookup table hanging around for all properties all the time; that's also a pretty big code size cost, as well as a security and secrecy liability.

The problem is that symbol lookup depends on the optional symbol table; this is stripped by strip on ELFish targets, and I believe is always stripped on Windows. This means that any symbol lookup for a symbol which does not participate in dynamic linking is going to fail to lookup, even on debug builds, particularly if you are deploying to a device (as strip often reduces the binary size ~50+%).

That's really no different from Apple platforms, where non-public Swift symbols get stripped when building for distribution too. Symbol table lookup is fundamentally not something to be relied on; it's fine for providing better debug info if it's there, but I don't think we should go out of our way to make it be there.

Sounds like you are against this change even if it’s only in debug builds on non-Mach-O platforms, is that correct?

I’ll do whatever you and @compnerd think is best. On the one hand, I think it would be a shame if Windows & Linux had a subpar debugging UX due to platform implementation details like this, but I personally don’t work on those platforms, and tbh I expect that figuring out how to do this is going to take me a long time, so I’m happy to abandon this aspect of the change. Also the non-ideal output is arguably an improvement over the status quo.

The one practical consideration is that if we don’t have a stable output from this function on Windows and Linux (because we’ll be printing memory addresses of the function, my understanding is that it would be different on every build), it may be difficult to unit test keypaths using FileCheck. I’m not super familiar with FileCheck, but the usage I saw in the existing unit tests for KeyPath looked like it just compares raw strings. Any suggestion on what to do with the existing test? Conditionally run it on MacOS only would be the obvious solution, I forget exactly what bug fix it was testing, but it seemed unlikely to work on one platform and break on another.

Definitely, the work you've already done is a huge improvement! Symbols on Linux or Windows being less readily available in debug builds is a bigger problem, I think, that also affects the quality of things like backtraces in error reports. It's worth improving for sure, but I don't think we need to address it specifically for what you've done for key paths.

Keep in mind that the output isn't stable even on macOS, so writing tests that rely on the output being a certain way, without also having mechanisms to require reflectability of the types, properties, and other declarations involved, is not a good idea. If a test is testing debug output specifically, you could use regexes in lit like {{bar|<fallback output>}} to match either form of output. Code that is testing whether a key path refers to certain properties should continue to use the key path == operator to do so.

It's worth improving for sure, but I don't think we need to address it specifically for what you've done for key paths.

Understood. I have changed the new tests in this pull request (and the one existing test that was affected by this change) to use regexes, as you suggested.

If there was ever a regression on MacOS/iOS that made symbol lookup fail, the tests would continue to succeed. But maybe that's a feature and not a bug.

I can go further and make these tests succeed even if both symbol lookup and metadata lookup fail, though I'm not sure if it's useful to do so.

Code that is testing whether a key path refers to certain properties should continue to use the key path == operator to do so

Agreed 100%. The issue I was describing was only with these tests that apply directly to AnyKeyPath.debugDescription. To your point about tests not relying on the output: I think these specific tests are the exception to that rule.