Adventures in Squeak: Do I need types? x64

Today I’ve been trying to address an unusual behaviour in my compiler during the process of porting resulting binary from x86 to x64.

One (of many) of the test binaries I create from the Squeak script GetServices.sq was failing for some reason. I needed to figure out why.

---

GetServices.sq

..snip..
struct {
  LPSTR             lpServiceName;  
  LPSTR             lpDisplayName;  
  DWORD             dwServiceType;  
  DWORD             dwCurrentState; 
  DWORD             dwControlsAccepted;
  DWORD             dwWin32ExitCode;
  DWORD             dwServiceSpecificExitCode;
  DWORD             dwCheckPoint;
  DWORD             dwWaitHint;
  DWORD             dwProcessId;
  DWORD             dwServiceFlags;
} SERVICESTRUCTURE

reserve psrv as * SERVICESTRUCTURE

define $SERVICE_WIN32_OWN_PROCESS 0x10
define $SERVICE_STATE_ALL 0x03


reserve nServices
reserve bb
reserve counter
reserve counterb


stdcall EnumServicesStatusExA ( hSCM, 0, $SERVICE_WIN32_OWN_PROCESS, $SERVICE_STATE_ALL, 0, 0, &bb, &nServices, 0, 0 )

cdecall wsprintfA (ptr, "Size of Buffer Required: '%d', nServices: '%d'.\n", bb, nServices) -> length
stdcall GetStdHandle ($STD_OUTPUT_HANDLE) -> hConsole
stdcall WriteConsole (hConsole, ptr, length, _, $NULL)

# Allocate the required memory

stdcall VirtualAlloc ($NULL, bb, $MEM_COMMIT_RESERVE, $PAGE_READWRITE) -> psrv

# Enumerate the Services
stdcall EnumServicesStatusExA ( hSCM, 0, $SERVICE_WIN32_OWN_PROCESS, $SERVICE_STATE_ALL, psrv, bb, &bb, &nServices, 0, 0 )

cdecall wsprintfA (ptr, "Size of Buffer Required: '%d', nServices: '%d'.\n", bb, nServices) -> length
stdcall GetStdHandle ($STD_OUTPUT_HANDLE) -> hConsole
stdcall WriteConsole (hConsole, ptr, length, _, $NULL)
reserve psrvBase
move psrv -> psrvBase

move 0 -> counter
do {
    cdecall wsprintfA (ptr, "lpServiceName: '%s'.\n", psrv>lpServiceName ) -> length
    stdcall WriteConsole (hConsole, ptr, length, _, $NULL)
    cdecall wsprintfA (ptr, "lpDisplayName: '%s'.\n", psrv>lpDisplayName ) -> length
    stdcall WriteConsole (hConsole, ptr, length, _, $NULL)
    cdecall wsprintfA (ptr, "PID: '%d'.\n", psrv>dwProcessId ) -> length
    stdcall WriteConsole (hConsole, ptr, length, _, $NULL)
    #Increment the pointer.
    add psrv 52
    add counter 1
} while( counter < nServices )

stdcall std-ExitProcess (0)

After compiling the binary and trying to run it, I ran into a fatal error somewhere after the two calls to WriteConsole.

At a surface level, the code makes two calls to the Advapi32.dll function EnumServiceStatusExA. The function is loosely defined here if you are interested. Essentially the first call is used to determine the size of the buffer required, in bytes. This is stored into the aptly titled ‘bb’ variable. The program then calls Kernel32.dll VirtualAlloc to request a buffer this size from the operating system.

This code worked at a functional level in x86 because of how Squeak viewed reserved variables. In x86 Squeak, every variable and symbol was assumed to be 0d04 bytes in width; registers are 0d04 bytes wide, variables are 0d04 bytes wide, and the values they store are, as far as Squeak cares, 0d04 bytes wide.

In x64 land, however, this is far less safe an assumption to make (NB: It was never a safe assumption to make; I wrote Squeak to work first and would fix things later. Hehe.).

In this instance, the operating system was returning 0d04 byte result passed from EnumServiceStatusExA to ‘bb’.

When x64 Squeak compiler tries to index this memory, however, it does not care for the width of the variable it received, resulting in the following compiled instruction:

mov <REGISTER>, qword ptr ss:<VARIABLE OFFSET>

Essentially, the Squeak compiler has placed the following 0d04 bytes into the 64 bit register.

This resulted in Squeak compiling a program that is now asking the operating system to provide 0d2.5 terabytes of memory.

The solution to this is pretty simple; Squeak presently performs a lookup for types now based on the ST_TYPE field I implemented a week back when migrating to x64. When reserving variables now in Squeak, I can optionally require the width of the symbol at compile time and adjust the offsets accordingly.

reserve bb will reserve 0d04 bytes on x86, 0d08 on x64.

reserve dword bb will reserve 0d04 bytes on x86, 0d04 on x64.

And with that, the day is saved.

Opening Handle to SCM.
Size of Buffer Required: '29448', nServices: '0'.
Size of Buffer Required: '0', nServices: '184'.
lpServiceName: 'AfVpnService'.
lpDisplayName: 'Bitdefender VPN core service'.
PID: '5868'.
lpServiceName: 'AJRouter'.
lpDisplayName: 'AllJoyn Router Service'.
PID: '0'.
lpServiceName: 'ALG'
..snip..