Security and DEP

[danrasband]

We just ran an Application Security Best Practices inspection using AWS Inspector (currently in preview), on our application, and it came up with the following "High" level notice:

The following executable files installed on Instance i-xxxxxxx do not support DEP: /home/ubuntu/.rbenv/versions/2.2.2/lib/ruby/gems/2.2.0/gems/skylight-0.8.0/lib/skylight/native/x86_64-linux/libskylight.so from Application Security Best Practices

Apparently, DEP stands for "Data Execution Prevention" and libskylight.so doesn't support it. I'm not sure if this is a Rust issue, or a skylight issue.

I'm not too concerned about the security warning, since I'm pretty confident that a Rust library is going to be much more secure than a C library, but it would be nice to figure it out and get the security alert to go away.

[wagenet]

@danrasband thanks for the heads up! I'm not a Rust guy so I'll leave it to others on the team to give a real response :)

[danrasband]

Actually, I'm not sure if this even has anything to do with Rust. 😕 All I know is that the native extension generated by this gem doesn't support DEP, according to AWS.

[bstrie]

Is there more documentation from AWS on what that warning actually means? We here in Rust-land can imagine several scenarios where something could be considered "not supporting DEP", but none are unambiguously what AWS is warning about and all seem to require some non-default configuration on Skylight's end.

[danrasband]

I'll see what I can dig up.

[danrasband]

I think this is the most we're going to get from AWS docs:

http://docs.aws.amazon.com/inspector/latest/userguide/inspector_application-security-best-practices.html

By the way, the "Stack Cookies Enabled" section is also an issue for libskylight.so.

[bstrie]

Yeah, those docs are about as unhelpful as possible. :P Is the tool open-source, so we can determine what it's looking for?

[danrasband]

I've added a question on AWS's support forum:

https://forums.aws.amazon.com/message.jspa?messageID=681927#681927

[bstrie]

Thanks!

[danrasband]

Does this comment help at all?

https://forums.aws.amazon.com/thread.jspa?messageID=682094#jive-message-holder

Specifically, it seems like there are possibly some compile options that could help:

it's a matter of compiling and linking with the relevant options (ie. stack-protector and nxcompat)

[bstrie]

AFAIK all Rust code should be compiled with those options by default. I'm not a Skylight employee but I doubt they'd have any reason to disable these in their build process. So what I'd like specifically is to understand what in the executable AWS is looking for, in the hopes that it's not hardcoded to detect something specific to C or C++ codegen.

[wycats]

I can confirm that as far as I can tell, we don't disable those options.

[bstrie]

@danrasband @wycats I've asked around today and determined today that currently Rust only uses stack cookies on Windows (support for other platforms is forthcoming). So that error isn't spurious. The DEP issue is still a mystery.

[danrasband]

Ah, very interesting!

[bstrie]

@danrasband For stack cookies, I believe the relevant tracking issue is rust-lang/rust#16012 . I've just left a comment to attempt to restart discussion in there, which appears to have stagnated for a while.

[wagenet]

Let me know if this is still an issue.

[ro31337]

@wagenet it is compliance issue. Some organizations can't use Skylight, because of automatic reports generated by tools like AWS. There is actually only one executable on multiple servers that has no DEP - skylight library. It would be really nice to have. May I ask you to reopen the issue to show that you're interested in fixing it? (I believe it's just Rust compiler flag that is not there, but will be added to Rust sooner or later).

[wagenet]

@ro31337 I understand your concern, but unfortunately, I'm not sure how to fix it. AFAICT, we aren't doing anything unusual in our build process or setting any strange flags. I'll investigate a little bit more to see if anything new comes up in my searches.

[wycats]

@ro31337 I'm rebooting this conversation with the Rust folks. It's an important effect for Rust-compiled code when used by security-conscious companies on AWS, and we (the Rust team) should figure out what to do about it.

[wycats]

@ro31337 We need some help reproducing this issue. Can you give us some more information on the environment you're running in?

What version and distribution of Linux are you running? What version of Ruby? Are you running inside a container?

[sfackler]

I think all you may need to do is add this to extconf.rb:

$LDFLAGS << " -Wl,-z,noexecstack"

[wagenet]

@sfackler thanks, I'm happy to give that a try if we can get a reproduction of it.

[sfackler]

I was able to reproduce on Arch Linux at least by just installing the skylight gem. readelf -l libskylight.so looks like this:

Elf file type is DYN (Shared object file)                                                                                                                                                                                                                                          
Entry point 0x53880                                                                                                                                                                                                                                                                
There are 6 program headers, starting at offset 64                                                                                                                                                                                                                                 
                                                                                                                                                                                                                                                                                   
Program Headers:                                                                                                                                                                                                                                                                   
  Type           Offset             VirtAddr           PhysAddr                                                                                                                                                                                                                    
                 FileSiz            MemSiz              Flags  Align                                                                                                                                                                                                               
  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000                                                                                                                                                                                                          
                 0x0000000000b7a254 0x0000000000b7a254  R E    0x200000                                                                                                                                                                                                            
  LOAD           0x0000000000b7a258 0x0000000000d7a258 0x0000000000d7a258                                                                                                                                                                                                          
                 0x0000000000040fa8 0x00000000000437a0  RW     0x200000                                                                                                                                                                                                            
  DYNAMIC        0x0000000000bb3138 0x0000000000db3138 0x0000000000db3138                                                                                                                                                                                                          
                 0x00000000000001e0 0x00000000000001e0  RW     0x8                                                                                                                                                                                                                 
  TLS            0x0000000000b7a258 0x0000000000d7a258 0x0000000000d7a258                                                                                                                                                                                                          
                 0x00000000000000f0 0x00000000000000f0  R      0x8                                                                                                                                                                                                                 
  GNU_EH_FRAME   0x0000000000af4220 0x0000000000af4220 0x0000000000af4220                                                                                                                                                                                                          
                 0x0000000000000008 0x0000000000000008  R      0x4                                                                                                                                                                                                                 
  GNU_STACK      0x0000000000000000 0x0000000000000000 0x0000000000000000                                                                                                                                                                                                          
                 0x0000000000000000 0x0000000000000000  RWE    0x8                                                                                                                                                                                                                 
                                                                                                                                                                                                                                                                                   
 Section to Segment mapping:                                                                                                                                                                                                                                                       
  Segment Sections...                                                                                                                                                                                                                                                              
   00     .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_r .rela.dyn .rela.plt .init .plt .text .fini .rodata .eh_frame_hdr .eh_frame .gcc_except_table                                                                                                                       
   01     .tdata .ctors .dtors .jcr .data.rel.ro .dynamic .got .got.plt .data .bss                                                                                                                                                                                                 
   02     .dynamic                                                                                                                                                                                                                                                                 
   03     .tdata                                                                                                                                                                                                                                                                   
   04     .eh_frame_hdr                                                                                                                                                                                                                                                            
   05   

In particular, the RWE in the GNU_STACK section means that it's asking for an executable stack.

[ro31337]

Steps to reproduce from our side:

• Create AWS EC2 instance with skylight library deployed (we use ruby, I think you can just put libskylight.so anywhere in your file system)

• Run proprietary tool Amazon Inspector against this instance with default rules.

• You should see the message (it can take some time to test your instance):

The following executable files on instance ... do not support DEP: /usr/local/rvm/gems/ruby-2.X.X@XXX/gems/skylight-1.3.0/lib/skylight/native/x86_64-linux/libskylight.so

@wycats please see above if this helps. There is no any special setup required here (I'd be happy to provide more steps if you're not successful reproducing that). I think @sfackler is right. The data segment for the binary file is executable: RWE.

[wycats]

@wagenet good news, we have a repro!