native-image fails with segfault in Docker container #624

renatoathaydes · 2018-08-20T18:06:31Z

Hi, I am doing lots of experiments with native-image and I wanted to see if I could get them running inside a tiny Docker image like scratch. I used the --static option so I thought it would not need anything pre-installed in the container image.

The native-image works fine in my Mac (without --static as it says it can't create static images on Mac).

Here's what happened when I tried to run my program in Docker:

[ [ SubstrateSegfaultHandler caught signal 11 ] ]

General Purpose Register Set Values: 

  RAX 00000000015de000
  RBX 0000000000969f60
  RCX 00007ffd2f6db3a0
  RDX 000000000001f8f0
  RBP 0000000000d38b20
  RSI 0000000000000001
  RDI 0000000000d38b20
  RSP 00007ffd2f6db210
  R8  0000000000000013
  R9  0000000000838698
  R10 000000000074a8c0
  R11 0000000000000012
  R12 0000000000000000
  R13 0000000000498f60
  R14 0000000000e02ec0
  R15 00000000015bdbe0
  EFL 0000000000010287
  RIP 0000000000723964
  

JavaFrameAnchor dump:

  Anchor 00007ffd2f6db388 LastJavaSP 00007ffd2f6db2b0
  
DeoptStubPointer address: 0000000000000000

TopFrame info:

  Lookup TotalFrameSize in CodeInfoTable:
  SourceTotalFrameSize -1
  Does not look like a Java Frame. Use JavaFrameAnchors to find LastJavaSP:
  Found matching Anchor:00007ffd2f6db388
  LastJavaSP 00007ffd2f6db2b0
  LastJavaIP 00000000004cb415
  
VMThreads info:

  VMThread 00000000015bdbe0  STATUS_IN_JAVA (safepoints disabled)  java.lang.Thread@0xce7b38
  
VM Thread State for current thread 00000000015bdbe0:

  0 (32 bytes): com.oracle.svm.core.genscavenge.ThreadLocalAllocation.pinnedTLAB = 
    00000000015bdbe0: 0000000000000000 0000000000000000
    00000000015bdbf0: 0000000000000000 0000000000000000
    
  
  32 (32 bytes): com.oracle.svm.core.genscavenge.ThreadLocalAllocation.regularTLAB = 
    00000000015bdc00: 00007ff587a00000 00007ff587b00000
    00000000015bdc10: 00007ff587a0f308 0000000000000000
    
  
  64 (8 bytes): com.oracle.svm.core.genscavenge.PinnedAllocatorImpl.openPinnedAllocator = null
  72 (8 bytes): com.oracle.svm.core.heap.NoAllocationVerifier.openVerifiers = null
  80 (8 bytes): com.oracle.svm.core.jdk.IdentityHashCodeSupport.hashCodeGeneratorTL = null
  88 (8 bytes): com.oracle.svm.core.snippets.SnippetRuntime.currentException = null
  96 (8 bytes): com.oracle.svm.core.thread.JavaThreads.currentThread = java.lang.Thread  0000000000ce7b38
  104 (8 bytes): com.oracle.svm.core.thread.ThreadingSupportImpl.activeTimer = null
  112 (8 bytes): com.oracle.svm.core.genscavenge.ThreadLocalAllocation.freeList = 0  0000000000000000
  120 (8 bytes): com.oracle.svm.core.posix.thread.PosixVMThreads.IsolateTL = 94598687557484880  0150150150150150
  128 (8 bytes): com.oracle.svm.core.stack.JavaFrameAnchors.lastAnchor = 140725399171976  00007ffd2f6db388
  136 (8 bytes): com.oracle.svm.core.thread.VMThreads.nextTL = 0  0000000000000000
  144 (4 bytes): com.oracle.svm.core.thread.Safepoint.safepointRequested = -21304  ffffacc8
  148 (4 bytes): com.oracle.svm.core.thread.Safepoint.safepointRequestedValueBeforeSafepoint = 0  00000000
  152 (4 bytes): com.oracle.svm.core.thread.VMOperationControl.isLockOwner = 0  00000000
  156 (4 bytes): com.oracle.svm.core.thread.VMThreads$StatusSupport.safepointsDisabledTL = 1  00000001
  160 (4 bytes): com.oracle.svm.core.thread.VMThreads$StatusSupport.statusTL = 1  00000001
  
VMOperation dump:

  No VMOperation in progress
  

RuntimeCodeCache dump:

  == [Recent RuntimeCodeCache operations: ]
  
  == [RuntimeCodeCache: 0 methods]
  
Deoptimizer dump:

  == [Recent Deoptimizer Events: 
  ]
  
Dump Counters:

  
Raw Stacktrace:

  00007ffd2f6db210: 00007ff587a0f178 00007ff587a0f128
  00007ffd2f6db220: 0000000000966fe8 00007ff587a0dc50
  00007ffd2f6db230: 00007ff500000001 000000000049d85f
  00007ffd2f6db240: 00000000009a70a8 0000000000000009
  00007ffd2f6db250: 00007ff587a0dbc0 00007ff587a0d920
  00007ffd2f6db260: 00007ff587a0dbe0 00007ff587a0f100
  00007ffd2f6db270: 00000000009633c8 00000000c0100702
  00007ffd2f6db280: 00000000015bdbe0 000000000074a8dd
  00007ffd2f6db290: 00000000004cb0b0 00000000007274c0
  00007ffd2f6db2a0: 0000000000c86018 00000000004cb415
  00007ffd2f6db2b0: 0000000000000000 0000000000000000
  00007ffd2f6db2c0: 00007ff587a0ccd0 b8339fa000000000
  00007ffd2f6db2d0: 0000000000000000 00000000fffffffe
  00007ffd2f6db2e0: 0000000000000000 0000000000000000
  00007ffd2f6db2f0: 0000000000974c90 00000000005c6dcd
  00007ffd2f6db300: 0000000100d369c0 0000000100000000
  00007ffd2f6db310: 00007ff587a0dc00 00000000005f54d5
  00007ffd2f6db320: 000000000097a1a0 fffffffe00000000
  00007ffd2f6db330: fffffffe00000000 0000000000000000
  00007ffd2f6db340: 00007ff587a0dc00 00007ff587a0dbd0
  00007ffd2f6db350: 00007ff587a0d940 000000000074a8c0
  00007ffd2f6db360: 00000000007274c0 0000000000ceae50
  00007ffd2f6db370: 00007ffd2f6db3a0 00007ffd2f6db3a8
  00007ffd2f6db380: 00007ff587a0ccd0 00007ffd2f6db2b0
  00007ffd2f6db390: 0000000000000000 00007ff587a0dbe0
  00007ffd2f6db3a0: 00007ff587a0d920 0000000200000002
  00007ffd2f6db3b0: 0000000000000000 0000000000000000
  00007ffd2f6db3c0: 0000000000000000 0000000000000000
  00007ffd2f6db3d0: 0000000000000000 00000000004cfb25
  00007ffd2f6db3e0: 00007ff587a0d538 00000000005f54d5
  00007ffd2f6db3f0: 0000000000d27288 00000000005a3778
  00007ffd2f6db400: 000000000098bad8 0000000100d272d8
  00007ffd2f6db410: 00007ff587a0d538 00000000005f7093
  00007ffd2f6db420: 0000000100d27288 0000000000ceadc0
  00007ffd2f6db430: 00007ff587a0d520 000000000040560a
  00007ffd2f6db440: 0000000000000000 0000000a0000002a
  00007ffd2f6db450: 0000000000d27288 00007ff587a0d520
  00007ffd2f6db460: 0000000000ceae50 0000000000818878
  00007ffd2f6db470: 0000000487a0cf98 0000000187a0c000
  00007ffd2f6db480: 00007ff587a0d920 0000000000000000
  00007ffd2f6db490: 0000000000000000 00007ff587a0ccd0
  00007ffd2f6db4a0: 00007ff587a0ccd0 00000000004d2095
  00007ffd2f6db4b0: 0000001587a0cf80 0000006c00000010
  00007ffd2f6db4c0: 00000000007b3328 00000000004715e4
  00007ffd2f6db4d0: 0000000100000020 0000000000000000
  00007ffd2f6db4e0: 00007ff587a0ccd0 00000000004d0dd3
  00007ffd2f6db4f0: c010070187a0d030 0000000000000011
  00007ffd2f6db500: 0000001187a0d3d0 00007ff587a0d0f0
  00007ffd2f6db510: 0000000487a0d408 00007ff587a0d0c0
  00007ffd2f6db520: 0000000187a0d408 000000000000006c
  00007ffd2f6db530: 00007ff587a08bc0 00000000005f54d5
  00007ffd2f6db540: 0000000000000000 00007ff587a0ccd0
  00007ffd2f6db550: 00007ff587a0d3d0 00000000004d239b
  00007ffd2f6db560: 00007ff587a08bc0 00000000005f7093
  00007ffd2f6db570: 0000000187a0c098 00007ff587a089d8
  00007ffd2f6db580: 00007ff587a0ccd0 00000000004d44f3
  00007ffd2f6db590: 00007ff587a0d288 0000000000000009
  00007ffd2f6db5a0: 0000000987a0d3d0 00007ff587a0cce8
  00007ffd2f6db5b0: 00007ff587a0ccd0 0000232800596288
  00007ffd2f6db5c0: 00007ff587a0d508 00000000004db918
  00007ffd2f6db5d0: 00007ff587a01560 0000000000595dd8
  00007ffd2f6db5e0: 0000000300000000 00007ff587a0d508
  00007ffd2f6db5f0: 0000232800000000 00007ff587a0ccd0
  00007ffd2f6db600: 00007ff587a0d4d8 000000000066d58e
  
Stacktrace Stage0:

  RSP 00007ffd2f6db2b0 RIP 00000000004cb415 FrameSize 304
  RSP 00007ffd2f6db3e0 RIP 00000000004cfb25 FrameSize 208
  RSP 00007ffd2f6db4b0 RIP 00000000004d2095 FrameSize 64
  RSP 00007ffd2f6db4f0 RIP 00000000004d0dd3 FrameSize 112
  RSP 00007ffd2f6db560 RIP 00000000004d239b FrameSize 48
  RSP 00007ffd2f6db590 RIP 00000000004d44f3 FrameSize 64
  RSP 00007ffd2f6db5d0 RIP 00000000004db918 FrameSize 64
  RSP 00007ffd2f6db610 RIP 000000000066d58e FrameSize 96
  RSP 00007ffd2f6db670 RIP 000000000066de95 FrameSize 80
  RSP 00007ffd2f6db6c0 RIP 0000000000403d7f FrameSize 64
  RSP 00007ffd2f6db700 RIP 0000000000403a54 FrameSize 64
  RSP 00007ffd2f6db740 RIP 00000000004034ce FrameSize 80
  RSP 00007ffd2f6db790 RIP 0000000000404321 FrameSize 64
  RSP 00007ffd2f6db7d0 RIP 000000000040aa33 FrameSize 144
  
Stacktrace Stage1:

  RSP 00007ffd2f6db2b0 RIP 00000000004cb415  com.oracle.svm.core.code.ImageCodeInfo@0x9627e0 name = image code
  RSP 00007ffd2f6db3e0 RIP 00000000004cfb25  com.oracle.svm.core.code.ImageCodeInfo@0x9627e0 name = image code
  RSP 00007ffd2f6db4b0 RIP 00000000004d2095  com.oracle.svm.core.code.ImageCodeInfo@0x9627e0 name = image code
  RSP 00007ffd2f6db4f0 RIP 00000000004d0dd3  com.oracle.svm.core.code.ImageCodeInfo@0x9627e0 name = image code
  RSP 00007ffd2f6db560 RIP 00000000004d239b  com.oracle.svm.core.code.ImageCodeInfo@0x9627e0 name = image code
  RSP 00007ffd2f6db590 RIP 00000000004d44f3  com.oracle.svm.core.code.ImageCodeInfo@0x9627e0 name = image code
  RSP 00007ffd2f6db5d0 RIP 00000000004db918  com.oracle.svm.core.code.ImageCodeInfo@0x9627e0 name = image code
  RSP 00007ffd2f6db610 RIP 000000000066d58e  com.oracle.svm.core.code.ImageCodeInfo@0x9627e0 name = image code
  RSP 00007ffd2f6db670 RIP 000000000066de95  com.oracle.svm.core.code.ImageCodeInfo@0x9627e0 name = image code
  RSP 00007ffd2f6db6c0 RIP 0000000000403d7f  com.oracle.svm.core.code.ImageCodeInfo@0x9627e0 name = image code
  RSP 00007ffd2f6db700 RIP 0000000000403a54  com.oracle.svm.core.code.ImageCodeInfo@0x9627e0 name = image code
  RSP 00007ffd2f6db740 RIP 00000000004034ce  com.oracle.svm.core.code.ImageCodeInfo@0x9627e0 name = image code
  RSP 00007ffd2f6db790 RIP 0000000000404321  com.oracle.svm.core.code.ImageCodeInfo@0x9627e0 name = image code
  RSP 00007ffd2f6db7d0 RIP 000000000040aa33  com.oracle.svm.core.code.ImageCodeInfo@0x9627e0 name = image code
  
Full Stacktrace:

  RSP 00007ffd2f6db2b0 RIP 00000000004cb415  [image code] com.oracle.svm.core.posix.headers.Netdb.freeaddrinfo(Netdb.java)
  RSP 00007ffd2f6db2b0 RIP 00000000004cb415  [image code] com.oracle.svm.core.posix.Target_java_net_Inet4AddressImpl.lookupAllHostAddr(PosixJavaNetSubstitutions.java:767)
  RSP 00007ffd2f6db3e0 RIP 00000000004cfb25  [image code] java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
  RSP 00007ffd2f6db3e0 RIP 00000000004cfb25  [image code] java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
  RSP 00007ffd2f6db4b0 RIP 00000000004d2095  [image code] java.net.InetAddress.getAllByName0(InetAddress.java:1276)
  RSP 00007ffd2f6db4f0 RIP 00000000004d0dd3  [image code] java.net.InetAddress.getAllByName(InetAddress.java:1192)
  RSP 00007ffd2f6db560 RIP 00000000004d239b  [image code] java.net.InetAddress.getAllByName(InetAddress.java:1126)
  RSP 00007ffd2f6db560 RIP 00000000004d239b  [image code] java.net.InetAddress.getByName(InetAddress.java:1076)
  RSP 00007ffd2f6db590 RIP 00000000004d44f3  [image code] java.net.InetSocketAddress.<init>(InetSocketAddress.java:220)
  RSP 00007ffd2f6db5d0 RIP 00000000004db918  [image code] java.net.Socket.<init>(Socket.java:211)
  RSP 00007ffd2f6db610 RIP 000000000066d58e  [image code] rawhttp.core.client.TcpRawHttpClient$DefaultOptions.getSocket(TcpRawHttpClient.java:178)
  RSP 00007ffd2f6db670 RIP 000000000066de95  [image code] rawhttp.core.client.TcpRawHttpClient.send(TcpRawHttpClient.java:71)
  RSP 00007ffd2f6db6c0 RIP 0000000000403d7f  [image code] com.athaydes.rawhttp.scraper.Main.scrape(Main.java:39)
  RSP 00007ffd2f6db700 RIP 0000000000403a54  [image code] com.athaydes.rawhttp.scraper.Main.run(Main.java:29)
  RSP 00007ffd2f6db740 RIP 00000000004034ce  [image code] com.athaydes.rawhttp.scraper.Main.main(Main.java:51)
  RSP 00007ffd2f6db790 RIP 0000000000404321  [image code] com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:177)
  RSP 00007ffd2f6db7d0 RIP 000000000040aa33  [image code] com.oracle.svm.core.code.CEntryPointCallStubs.com_002eoracle_002esvm_002ecore_002eJavaMainWrapper_002erun_0028int_002corg_002egraalvm_002enativeimage_002ec_002etype_002eCCharPointerPointer_0029(generated:0)
  
Use runtime option -R:-InstallSegfaultHandler if you don't want to use SubstrateSegfaultHandler.

Bye bye ...

I also tried using alpine as a base image, but it also got an error:

*** Error in `/scraper': free(): invalid pointer: 0x000000000096a2e8 ***
======= Backtrace: =========
[0x71d877]
[0x723102]
[0x72391e]
[0x74a8dd]
[0x4cb415]
======= Memory map: ========
00400000-00a86000 r-xp 00000000 08:01 2490717                            /scraper
00c85000-00d3a000 rwxp 00685000 08:01 2490717                            /scraper
00d3a000-00d41000 rwxp 00000000 00:00 0 
02034000-02057000 rwxp 00000000 00:00 0                                  [heap]
7fa1f0000000-7fa1f0023000 rwxp 00000000 00:00 0 
7fa1f0023000-7fa1f4000000 ---p 00000000 00:00 0 
7fa1f5400000-7fa1f5500000 rwxp 00000000 00:00 0 
7fa1f553f000-7fa1f5540000 rwxp 00000000 00:00 0 
7ffed7ba5000-7ffed7bc6000 rwxp 00000000 00:00 0                          [stack]
7ffed7bd4000-7ffed7bd6000 r--p 00000000 00:00 0                          [vvar]
7ffed7bd6000-7ffed7bd8000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
[ [ SubstrateSegfaultHandler caught signal 11 ] ]

General Purpose Register Set Values: 

  RAX 0000000000000006
  RBX 0000000000000049
  RCX 0000000000719bae
  RDX 0000000000000000
  RBP 00007ffed7bc36b0
  RSI 00007ffed7bc3210
  RDI 0000000000000002
  RSP 00007ffed7bc3320
  R8  0000000000000000
  R9  0000000000000006
  R10 0000000000000008
  R11 0000000000000246
  R12 0000000000000049
  R13 00007ffed7bc34c0
  R14 00007ffed7bc34c0
  R15 00007fa1f553f000
  EFL 0000000000010246
  RIP 0000000000719ec9
  

JavaFrameAnchor dump:

  No anchors
  
DeoptStubPointer address: 0000000000000000

TopFrame info:

  Lookup TotalFrameSize in CodeInfoTable:
  SourceTotalFrameSize -1
  Does not look like a Java Frame. Use JavaFrameAnchors to find LastJavaSP:
  
VMThreads info:

  VMThread 0000000002036be0  STATUS_IN_NATIVE  java.lang.Thread@0xce7b38
  
VM Thread State for current thread 00007fa1f553f000:

  0 (32 bytes): com.oracle.svm.core.genscavenge.ThreadLocalAllocation.pinnedTLAB = 
    00007fa1f553f000: 202a2a2a00001000 6e6920726f727245
    00007fa1f553f010: 70617263732f6020 657266203a277265
    
  
  32 (32 bytes): com.oracle.svm.core.genscavenge.ThreadLocalAllocation.regularTLAB = 
    00007fa1f553f020: 766e69203a292865 696f702064696c61
    00007fa1f553f030: 7830203a7265746e 3030303030303030
    
  
  64 (8 bytes): com.oracle.svm.core.genscavenge.PinnedAllocatorImpl.openPinnedAllocator =

This is the project I am using for tests:

https://github.com/renatoathaydes/rawhttp-scraper/tree/use-graal-js-api/rawhttp-scraper

I tagged the commit with docker-experiment so you can check it out easily, if needed.

I just go to this folder, run docker build -t scraper ., then execute it with docker run -it scraper get-speakers (it will try to send a HTTP request to http://localhost:9000/java-meetup/speakers which I know, will not work unless you expose a server in localhost on the container, but I still expected to just see a ConnectionException or something similar).

A command that does work (as it just prints help to sysout) is docker run -it scraper -h, in case you need to test that.

The text was updated successfully, but these errors were encountered:

renatoathaydes · 2018-08-20T18:22:05Z

I changed the base Docker image to:

FROM debian:stable-slim

Which is the same as the one used by the graal image.... and it seems to work (no segfault). So, not sure if this is a bug or I was just using an incompatible kernel?! If so, sorry for the trouble.

renatoathaydes · 2018-08-20T19:27:07Z

By executing a shell in the Docker container and trying some different requests, it's pretty easy to cause a segfault.

For example, start a container shell with:

docker run --entrypoint /bin/bash -it scraper

Then, create a simple request in a local file, then run the "scraper" with that:

$ echo "
> GET http://ip.jsontest.com/
> Accept: application/json
> " > req.txt
$ ./scraper req.txt 
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Content-Type: application/json; charset=ISO-8859-1
X-Cloud-Trace-Context: 0dd86ba705542aa4deccf40e031edd24
Date: Mon, 20 Aug 2018 19:23:10 GMT
Server: Google Frontend
Content-Length: 24

{"ip": "????"}

Now, try a request to a non-existent server:

$ echo "
GET http://error.jsontest.org/
Accept: application/json
" > req.txt
$ ./scraper req.txt 
*** Error in `./scraper': munmap_chunk(): invalid pointer: 0x0000000000969658 ***
======= Backtrace: =========
[0x71d877]
[0x723102]
[0x74a8dd]
[0x4cb415]
======= Memory map: ========
00400000-00a86000 r-xp 00000000 08:01 2490717                            /scraper
00c85000-00d3a000 rwxp 00685000 08:01 2490717                            /scraper
00d3a000-00d41000 rwxp 00000000 00:00 0 
019ab000-019ce000 rwxp 00000000 00:00 0                                  [heap]
7f35d3a0c000-7f35d3a20000 r-xp 00000000 08:01 2102401                    /lib/x86_64-linux-gnu/libresolv-2.24.so
7f35d3a20000-7f35d3c1f000 ---p 00014000 08:01 2102401                    /lib/x86_64-linux-gnu/libresolv-2.24.so
7f35d3c1f000-7f35d3c20000 r-xp 00013000 08:01 2102401                    /lib/x86_64-linux-gnu/libresolv-2.24.so
7f35d3c20000-7f35d3c21000 rwxp 00014000 08:01 2102401                    /lib/x86_64-linux-gnu/libresolv-2.24.so
7f35d3c21000-7f35d3c23000 rwxp 00000000 00:00 0 
7f35d3c23000-7f35d3c28000 r-xp 00000000 08:01 2102380                    /lib/x86_64-linux-gnu/libnss_dns-2.24.so
7f35d3c28000-7f35d3e27000 ---p 00005000 08:01 2102380                    /lib/x86_64-linux-gnu/libnss_dns-2.24.so
7f35d3e27000-7f35d3e28000 r-xp 00004000 08:01 2102380                    /lib/x86_64-linux-gnu/libnss_dns-2.24.so
7f35d3e28000-7f35d3e29000 rwxp 00005000 08:01 2102380                    /lib/x86_64-linux-gnu/libnss_dns-2.24.so
7f35d3e29000-7f35d3f29000 rwxp 00000000 00:00 0 
7f35d3f29000-7f35d3f4c000 r-xp 00000000 08:01 2102324                    /lib/x86_64-linux-gnu/ld-2.24.so
7f35d3f4c000-7f35d414c000 ---p 00023000 08:01 2102324                    /lib/x86_64-linux-gnu/ld-2.24.so
7f35d414c000-7f35d414d000 r-xp 00023000 08:01 2102324                    /lib/x86_64-linux-gnu/ld-2.24.so
7f35d414d000-7f35d414e000 rwxp 00024000 08:01 2102324                    /lib/x86_64-linux-gnu/ld-2.24.so
7f35d414e000-7f35d414f000 rwxp 00000000 00:00 0 
7f35d414f000-7f35d42e4000 r-xp 00000000 08:01 2102342                    /lib/x86_64-linux-gnu/libc-2.24.so
7f35d42e4000-7f35d44e4000 ---p 00195000 08:01 2102342                    /lib/x86_64-linux-gnu/libc-2.24.so
7f35d44e4000-7f35d44e8000 r-xp 00195000 08:01 2102342                    /lib/x86_64-linux-gnu/libc-2.24.so
7f35d44e8000-7f35d44ea000 rwxp 00199000 08:01 2102342                    /lib/x86_64-linux-gnu/libc-2.24.so
7f35d44ea000-7f35d44ee000 rwxp 00000000 00:00 0 
7f35d44ee000-7f35d44f8000 r-xp 00000000 08:01 2102382                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7f35d44f8000-7f35d46f8000 ---p 0000a000 08:01 2102382                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7f35d46f8000-7f35d46f9000 r-xp 0000a000 08:01 2102382                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7f35d46f9000-7f35d46fa000 rwxp 0000b000 08:01 2102382                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7f35d46fa000-7f35d4800000 rwxp 00000000 00:00 0 
7f35d4857000-7f35d4858000 rwxp 00000000 00:00 0 
7fff47b34000-7fff47b55000 rwxp 00000000 00:00 0                          [stack]
7fff47bf8000-7fff47bfa000 r--p 00000000 00:00 0                          [vvar]
7fff47bfa000-7fff47bfc000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted

So, seems to be an actual bug.

olpaw · 2018-08-21T07:33:54Z

I changed the base Docker image to: FROM debian:stable-slim Which is the same as the one used by the graal image.... and it seems to work (no segfault)

If you are using a native-image in docker you need to ensure that you base your docker image on an image that uses glibc as its standard C library. This is the case for debian:stable-slim but not for scratch (no libc) or alpine (musl).

olpaw · 2018-08-21T07:35:36Z

By executing a shell in the Docker container and trying some different requests, it's pretty easy to cause a segfault.

Can you remove --static in line https://github.com/renatoathaydes/rawhttp-scraper/blob/3dbf51eca64eb9b9c5bd69157250a1450aa73e3d/rawhttp-scraper/Dockerfile#L14 and try again?

renatoathaydes · 2018-08-21T12:06:16Z

Can you remove --static

Funny that you mention that :) Just tried doing it and got the same error when I make a request to a server that can't be found (used host jsontest.org instead of the existing jsontest.com):

root@446c849c97ca:/# echo "
GET http://jsontest.org
Accept: application/json
User-Agent: RawHTTP
" | ./rawhttp send
*** Error in `./rawhttp': munmap_chunk(): invalid pointer: 0x000055b8321738e8 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x70bfb)[0x7f2a9c7e9bfb]
/lib/x86_64-linux-gnu/libc.so.6(+0x76fc6)[0x7f2a9c7effc6]
/lib/x86_64-linux-gnu/libc.so.6(freeaddrinfo+0x1d)[0x7f2a9c84e1bd]
./rawhttp(+0x384d55)[0x55b831d14d55]
======= Memory map: ========
55b831990000-55b832274000 r-xp 00000000 08:01 2755876                    /rawhttp
55b832474000-55b832475000 r-xp 008e4000 08:01 2755876                    /rawhttp
55b832475000-55b832553000 rwxp 008e5000 08:01 2755876                    /rawhttp
55b8344ed000-55b83450e000 rwxp 00000000 00:00 0                          [heap]
7f2a9bdba000-7f2a9bdd0000 r-xp 00000000 08:01 2102360                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f2a9bdd0000-7f2a9bfcf000 ---p 00016000 08:01 2102360                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f2a9bfcf000-7f2a9bfd0000 r-xp 00015000 08:01 2102360                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f2a9bfd0000-7f2a9bfd1000 rwxp 00016000 08:01 2102360                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7f2a9bfd1000-7f2a9bfe5000 r-xp 00000000 08:01 2102401                    /lib/x86_64-linux-gnu/libresolv-2.24.so
7f2a9bfe5000-7f2a9c1e4000 ---p 00014000 08:01 2102401                    /lib/x86_64-linux-gnu/libresolv-2.24.so
7f2a9c1e4000-7f2a9c1e5000 r-xp 00013000 08:01 2102401                    /lib/x86_64-linux-gnu/libresolv-2.24.so
7f2a9c1e5000-7f2a9c1e6000 rwxp 00014000 08:01 2102401                    /lib/x86_64-linux-gnu/libresolv-2.24.so
7f2a9c1e6000-7f2a9c1e8000 rwxp 00000000 00:00 0 
7f2a9c1e8000-7f2a9c1ed000 r-xp 00000000 08:01 2102380                    /lib/x86_64-linux-gnu/libnss_dns-2.24.so
7f2a9c1ed000-7f2a9c3ec000 ---p 00005000 08:01 2102380                    /lib/x86_64-linux-gnu/libnss_dns-2.24.so
7f2a9c3ec000-7f2a9c3ed000 r-xp 00004000 08:01 2102380                    /lib/x86_64-linux-gnu/libnss_dns-2.24.so
7f2a9c3ed000-7f2a9c3ee000 rwxp 00005000 08:01 2102380                    /lib/x86_64-linux-gnu/libnss_dns-2.24.so
7f2a9c3ee000-7f2a9c3f8000 r-xp 00000000 08:01 2102382                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7f2a9c3f8000-7f2a9c5f8000 ---p 0000a000 08:01 2102382                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7f2a9c5f8000-7f2a9c5f9000 r-xp 0000a000 08:01 2102382                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7f2a9c5f9000-7f2a9c5fa000 rwxp 0000b000 08:01 2102382                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7f2a9c5fa000-7f2a9c700000 rwxp 00000000 00:00 0 
7f2a9c779000-7f2a9c90e000 r-xp 00000000 08:01 2102342                    /lib/x86_64-linux-gnu/libc-2.24.so
7f2a9c90e000-7f2a9cb0e000 ---p 00195000 08:01 2102342                    /lib/x86_64-linux-gnu/libc-2.24.so
7f2a9cb0e000-7f2a9cb12000 r-xp 00195000 08:01 2102342                    /lib/x86_64-linux-gnu/libc-2.24.so
7f2a9cb12000-7f2a9cb14000 rwxp 00199000 08:01 2102342                    /lib/x86_64-linux-gnu/libc-2.24.so
7f2a9cb14000-7f2a9cb18000 rwxp 00000000 00:00 0 
7f2a9cb18000-7f2a9cb20000 r-xp 00000000 08:01 2102350                    /lib/x86_64-linux-gnu/libcrypt-2.24.so
7f2a9cb20000-7f2a9cd20000 ---p 00008000 08:01 2102350                    /lib/x86_64-linux-gnu/libcrypt-2.24.so
7f2a9cd20000-7f2a9cd21000 r-xp 00008000 08:01 2102350                    /lib/x86_64-linux-gnu/libcrypt-2.24.so
7f2a9cd21000-7f2a9cd22000 rwxp 00009000 08:01 2102350                    /lib/x86_64-linux-gnu/libcrypt-2.24.so
7f2a9cd22000-7f2a9cd50000 rwxp 00000000 00:00 0 
7f2a9cd50000-7f2a9cd57000 r-xp 00000000 08:01 2102403                    /lib/x86_64-linux-gnu/librt-2.24.so
7f2a9cd57000-7f2a9cf56000 ---p 00007000 08:01 2102403                    /lib/x86_64-linux-gnu/librt-2.24.so
7f2a9cf56000-7f2a9cf57000 r-xp 00006000 08:01 2102403                    /lib/x86_64-linux-gnu/librt-2.24.so
7f2a9cf57000-7f2a9cf58000 rwxp 00007000 08:01 2102403                    /lib/x86_64-linux-gnu/librt-2.24.so
7f2a9cf58000-7f2a9cf71000 r-xp 00000000 08:01 2102424                    /lib/x86_64-linux-gnu/libz.so.1.2.8
7f2a9cf71000-7f2a9d170000 ---p 00019000 08:01 2102424                    /lib/x86_64-linux-gnu/libz.so.1.2.8
7f2a9d170000-7f2a9d171000 r-xp 00018000 08:01 2102424                    /lib/x86_64-linux-gnu/libz.so.1.2.8
7f2a9d171000-7f2a9d172000 rwxp 00019000 08:01 2102424                    /lib/x86_64-linux-gnu/libz.so.1.2.8
7f2a9d172000-7f2a9d18a000 r-xp 00000000 08:01 2102399                    /lib/x86_64-linux-gnu/libpthread-2.24.so
7f2a9d18a000-7f2a9d389000 ---p 00018000 08:01 2102399                    /lib/x86_64-linux-gnu/libpthread-2.24.so
7f2a9d389000-7f2a9d38a000 r-xp 00017000 08:01 2102399                    /lib/x86_64-linux-gnu/libpthread-2.24.so
7f2a9d38a000-7f2a9d38b000 rwxp 00018000 08:01 2102399                    /lib/x86_64-linux-gnu/libpthread-2.24.so
7f2a9d38b000-7f2a9d38f000 rwxp 00000000 00:00 0 
7f2a9d38f000-7f2a9d392000 r-xp 00000000 08:01 2102352                    /lib/x86_64-linux-gnu/libdl-2.24.so
7f2a9d392000-7f2a9d591000 ---p 00003000 08:01 2102352                    /lib/x86_64-linux-gnu/libdl-2.24.so
7f2a9d591000-7f2a9d592000 r-xp 00002000 08:01 2102352                    /lib/x86_64-linux-gnu/libdl-2.24.so
7f2a9d592000-7f2a9d593000 rwxp 00003000 08:01 2102352                    /lib/x86_64-linux-gnu/libdl-2.24.so
7f2a9d593000-7f2a9d5b6000 r-xp 00000000 08:01 2102324                    /lib/x86_64-linux-gnu/ld-2.24.so
7f2a9d7ad000-7f2a9d7b1000 rwxp 00000000 00:00 0 
7f2a9d7b2000-7f2a9d7b6000 rwxp 00000000 00:00 0 
7f2a9d7b6000-7f2a9d7b7000 r-xp 00023000 08:01 2102324                    /lib/x86_64-linux-gnu/ld-2.24.so
7f2a9d7b7000-7f2a9d7b8000 rwxp 00024000 08:01 2102324                    /lib/x86_64-linux-gnu/ld-2.24.so
7f2a9d7b8000-7f2a9d7b9000 rwxp 00000000 00:00 0 
7ffc9a543000-7ffc9a564000 rwxp 00000000 00:00 0                          [stack]
7ffc9a599000-7ffc9a59b000 r--p 00000000 00:00 0                          [vvar]
7ffc9a59b000-7ffc9a59d000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted

Requests to servers that exist work fine. In my Mac (without Docker), the same request (also running a native-image) would result in:

java.net.UnknownHostException: jsontest.org

So I think this is the case that the native image in Docker is not handling correctly.

renatoathaydes · 2018-08-21T12:10:58Z

If you are using a native-image in docker you need to ensure that you base your docker image on an image that uses glibc as its standard C library.

Is this the case even when I use --static to create the native image?

olpaw · 2018-08-21T12:35:27Z

Is this the case even when I use --static to create the native image?

It depends. See #571 (comment)

graemerocher · 2018-09-06T10:16:26Z

I am also seeing this bug or something similar when attempting to build an image for Micronaut using docker. Trace is:

docker run -p 8080:8080 test-graal:latest
*** Error in `/app/test-graal': double free or corruption (out): 0x00005606514eb1c0 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x70bfb)[0x7fd783921bfb]
/lib/x86_64-linux-gnu/libc.so.6(+0x76fc6)[0x7fd783927fc6]
/lib/x86_64-linux-gnu/libc.so.6(+0x7780e)[0x7fd78392880e]
/lib/x86_64-linux-gnu/libc.so.6(freeaddrinfo+0x1d)[0x7fd7839861bd]
/app/test-graal(+0x16d2335)[0x5606505d2335]
======= Memory map: ========
56064ef00000-560651be0000 r-xp 00000000 08:01 2235179                    /app/test-graal
560651ddf000-560651de0000 r-xp 02cdf000 08:01 2235179                    /app/test-graal
560651de0000-560651fc7000 rwxp 02ce0000 08:01 2235179                    /app/test-graal
560652690000-5606526b1000 rwxp 00000000 00:00 0                          [heap]
7fd774000000-7fd774021000 rwxp 00000000 00:00 0 
7fd774021000-7fd778000000 ---p 00000000 00:00 0 
7fd77c000000-7fd77c021000 rwxp 00000000 00:00 0 
7fd77c021000-7fd780000000 ---p 00000000 00:00 0 
7fd7824ba000-7fd7824d0000 r-xp 00000000 08:01 1326101                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fd7824d0000-7fd7826cf000 ---p 00016000 08:01 1326101                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fd7826cf000-7fd7826d0000 r-xp 00015000 08:01 1326101                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fd7826d0000-7fd7826d1000 rwxp 00016000 08:01 1326101                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fd7826d1000-7fd7826e5000 r-xp 00000000 08:01 1326236                    /lib/x86_64-linux-gnu/libresolv-2.24.so
7fd7826e5000-7fd7828e4000 ---p 00014000 08:01 1326236                    /lib/x86_64-linux-gnu/libresolv-2.24.so
7fd7828e4000-7fd7828e5000 r-xp 00013000 08:01 1326236                    /lib/x86_64-linux-gnu/libresolv-2.24.so
7fd7828e5000-7fd7828e6000 rwxp 00014000 08:01 1326236                    /lib/x86_64-linux-gnu/libresolv-2.24.so
7fd7828e6000-7fd7828e8000 rwxp 00000000 00:00 0 
7fd7828e8000-7fd7828ed000 r-xp 00000000 08:01 1326121                    /lib/x86_64-linux-gnu/libnss_dns-2.24.so
7fd7828ed000-7fd782aec000 ---p 00005000 08:01 1326121                    /lib/x86_64-linux-gnu/libnss_dns-2.24.so
7fd782aec000-7fd782aed000 r-xp 00004000 08:01 1326121                    /lib/x86_64-linux-gnu/libnss_dns-2.24.so
7fd782aed000-7fd782aee000 rwxp 00005000 08:01 1326121                    /lib/x86_64-linux-gnu/libnss_dns-2.24.so
7fd782aee000-7fd782af8000 r-xp 00000000 08:01 1326123                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7fd782af8000-7fd782cf8000 ---p 0000a000 08:01 1326123                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7fd782cf8000-7fd782cf9000 r-xp 0000a000 08:01 1326123                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7fd782cf9000-7fd782cfa000 rwxp 0000b000 08:01 1326123                    /lib/x86_64-linux-gnu/libnss_files-2.24.so
7fd782cfa000-7fd782e00000 rwxp 00000000 00:00 0 
7fd782eff000-7fd782f00000 ---p 00000000 00:00 0 
7fd782f00000-7fd783800000 rwxp 00000000 00:00 0 
7fd7838b1000-7fd783a46000 r-xp 00000000 08:01 1326083                    /lib/x86_64-linux-gnu/libc-2.24.so
7fd783a46000-7fd783c46000 ---p 00195000 08:01 1326083                    /lib/x86_64-linux-gnu/libc-2.24.so
7fd783c46000-7fd783c4a000 r-xp 00195000 08:01 1326083                    /lib/x86_64-linux-gnu/libc-2.24.so
7fd783c4a000-7fd783c4c000 rwxp 00199000 08:01 1326083                    /lib/x86_64-linux-gnu/libc-2.24.so
7fd783c4c000-7fd783c50000 rwxp 00000000 00:00 0 
7fd783c50000-7fd783c57000 r-xp 00000000 08:01 1326238                    /lib/x86_64-linux-gnu/librt-2.24.so
7fd783c57000-7fd783e56000 ---p 00007000 08:01 1326238                    /lib/x86_64-linux-gnu/librt-2.24.so
7fd783e56000-7fd783e57000 r-xp 00006000 08:01 1326238                    /lib/x86_64-linux-gnu/librt-2.24.so
7fd783e57000-7fd783e58000 rwxp 00007000 08:01 1326238                    /lib/x86_64-linux-gnu/librt-2.24.so
7fd783e58000-7fd783e60000 r-xp 00000000 08:01 1326091                    /lib/x86_64-linux-gnu/libcrypt-2.24.so
7fd783e60000-7fd784060000 ---p 00008000 08:01 1326091                    /lib/x86_64-linux-gnu/libcrypt-2.24.so
7fd784060000-7fd784061000 r-xp 00008000 08:01 1326091                    /lib/x86_64-linux-gnu/libcrypt-2.24.so
7fd784061000-7fd784062000 rwxp 00009000 08:01 1326091                    /lib/x86_64-linux-gnu/libcrypt-2.24.so
7fd784062000-7fd784090000 rwxp 00000000 00:00 0 
7fd784090000-7fd7840a9000 r-xp 00000000 08:01 1326259                    /lib/x86_64-linux-gnu/libz.so.1.2.8
7fd7840a9000-7fd7842a8000 ---p 00019000 08:01 1326259                    /lib/x86_64-linux-gnu/libz.so.1.2.8
7fd7842a8000-7fd7842a9000 r-xp 00018000 08:01 1326259                    /lib/x86_64-linux-gnu/libz.so.1.2.8
7fd7842a9000-7fd7842aa000 rwxp 00019000 08:01 1326259                    /lib/x86_64-linux-gnu/libz.so.1.2.8
7fd7842aa000-7fd7842ad000 r-xp 00000000 08:01 1326093                    /lib/x86_64-linux-gnu/libdl-2.24.so
7fd7842ad000-7fd7844ac000 ---p 00003000 08:01 1326093                    /lib/x86_64-linux-gnu/libdl-2.24.so
7fd7844ac000-7fd7844ad000 r-xp 00002000 08:01 1326093                    /lib/x86_64-linux-gnu/libdl-2.24.so
7fd7844ad000-7fd7844ae000 rwxp 00003000 08:01 1326093                    /lib/x86_64-linux-gnu/libdl-2.24.so
7fd7844ae000-7fd7844c6000 r-xp 00000000 08:01 1326234                    /lib/x86_64-linux-gnu/libpthread-2.24.so
7fd7844c6000-7fd7846c5000 ---p 00018000 08:01 1326234                    /lib/x86_64-linux-gnu/libpthread-2.24.so
7fd7846c5000-7fd7846c6000 r-xp 00017000 08:01 1326234                    /lib/x86_64-linux-gnu/libpthread-2.24.so
7fd7846c6000-7fd7846c7000 rwxp 00018000 08:01 1326234                    /lib/x86_64-linux-gnu/libpthread-2.24.so
7fd7846c7000-7fd7846cb000 rwxp 00000000 00:00 0 
7fd7846cb000-7fd7846ee000 r-xp 00000000 08:01 1326065                    /lib/x86_64-linux-gnu/ld-2.24.so
7fd7848e5000-7fd7848ea000 rwxp 00000000 00:00 0 
7fd7848ea000-7fd7848eb000 rw-s 00000000 00:73 70127                      /dev/shm/2iOyY3 (deleted)
7fd7848eb000-7fd7848ee000 rwxp 00000000 00:00 0 
7fd7848ee000-7fd7848ef000 r-xp 00023000 08:01 1326065                    /lib/x86_64-linux-gnu/ld-2.24.so
7fd7848ef000-7fd7848f0000 rwxp 00024000 08:01 1326065                    /lib/x86_64-linux-gnu/ld-2.24.so
7fd7848f0000-7fd7848f1000 rwxp 00000000 00:00 0 
7ffff6a49000-7ffff6a6a000 rwxp 00000000 00:00 0                          [stack]
7ffff6aac000-7ffff6aae000 r--p 00000000 00:00 0                          [vvar]
7ffff6aae000-7ffff6ab0000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
[ [ SubstrateSegfaultHandler caught signal 11 ] ]

General Purpose Register Set Values: 

  RAX 0000000000000006
  RBX 0000000000000058
  RCX 00007fd7838e3fff
  RDX 0000000000000000
  RBP 00007ffff6a674a0
  RSI 00007ffff6a67000
  RDI 0000000000000002
  RSP 00007ffff6a67110
  R8  0000000000000000
  R9  00007ffff6a67000
  R10 0000000000000008
  R11 0000000000000246
  R12 0000000000000058
  R13 00007ffff6a672b0
  R14 00007ffff6a672b0
  R15 00007fd7848e9000
  EFL 0000000000010246
  RIP 00007fd7838e5529
  

JavaFrameAnchor dump:

  No anchors
  
DeoptStubPointer address: 0000000000000000

TopFrame info:

  Lookup TotalFrameSize in CodeInfoTable:
  SourceTotalFrameSize -1
  Does not look like a Java Frame. Use JavaFrameAnchors to find LastJavaSP:
  
VMThreads info:

  VMThread 00007fd77c0008c0  STATUS_IN_NATIVE  java.lang.Thread@0x7fd783701a58
  VMThread 0000560652690010  STATUS_IN_NATIVE  java.lang.Thread@0x560651e82290
  
VM Thread State for current thread 00007fd7848e9000:

  0 (32 bytes): com.oracle.svm.core.genscavenge.ThreadLocalAllocation.pinnedTLAB = 
    00007fd7848e9000: 202a2a2a00001000 6e6920726f727245
    00007fd7848e9010: 742f7070612f6020 616172672d747365
    
  
  32 (32 bytes): com.oracle.svm.core.genscavenge.ThreadLocalAllocation.regularTLAB = 
    00007fd7848e9020: 62756f64203a276c 206565726620656c
    00007fd7848e9030: 7572726f6320726f 6f28206e6f697470
    
  
  64 (8 bytes): com.oracle.svm.core.genscavenge.PinnedAllocatorImpl.openPinnedAllocator = Graemes-iMac-Pro:test-graal

cstancu added the native-image label Aug 20, 2018

cstancu assigned olpaw Aug 20, 2018

raboof mentioned this issue Apr 10, 2019

InetAddress.getLocalHost throws UnknownHostException on Alpine #1151

Closed

andreaTP mentioned this issue Nov 26, 2019

Fix SSL Hanshake and DNS issues codacy/codacy-coverage-reporter#146

Merged

olpaw closed this as completed Dec 19, 2019

github-actions bot mentioned this issue Jan 18, 2021

Broken link report codacy/docs#407

Closed

github-actions bot mentioned this issue Feb 22, 2021

Broken link report codacy/docs#480

Closed

github-actions bot mentioned this issue Dec 15, 2021

Broken link report codacy/docs#954

Closed

github-actions bot mentioned this issue Jul 28, 2022

Broken link report codacy/docs#1332

Closed

github-actions bot mentioned this issue Aug 22, 2022

Broken link report codacy/docs#1367

Closed

github-actions bot mentioned this issue Sep 26, 2022

Broken link report codacy/docs#1422

Closed

github-actions bot mentioned this issue Oct 24, 2022

Broken link report codacy/docs#1478

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

native-image fails with segfault in Docker container #624

native-image fails with segfault in Docker container #624

renatoathaydes commented Aug 20, 2018

renatoathaydes commented Aug 20, 2018

Uh oh!

renatoathaydes commented Aug 20, 2018

Uh oh!

olpaw commented Aug 21, 2018

Uh oh!

olpaw commented Aug 21, 2018

Uh oh!

renatoathaydes commented Aug 21, 2018

Uh oh!

renatoathaydes commented Aug 21, 2018

Uh oh!

olpaw commented Aug 21, 2018

Uh oh!

graemerocher commented Sep 6, 2018

Uh oh!

native-image fails with segfault in Docker container #624

native-image fails with segfault in Docker container #624

Comments

renatoathaydes commented Aug 20, 2018

renatoathaydes commented Aug 20, 2018

Uh oh!

renatoathaydes commented Aug 20, 2018

Uh oh!

olpaw commented Aug 21, 2018

Uh oh!

olpaw commented Aug 21, 2018

Uh oh!

renatoathaydes commented Aug 21, 2018

Uh oh!

renatoathaydes commented Aug 21, 2018

Uh oh!

olpaw commented Aug 21, 2018

Uh oh!

graemerocher commented Sep 6, 2018

Uh oh!