Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSX support tracking issue #152

Closed
7 of 11 tasks
dataf3l opened this issue Oct 3, 2018 · 40 comments
Closed
7 of 11 tasks

OSX support tracking issue #152

dataf3l opened this issue Oct 3, 2018 · 40 comments
Labels
BIG-TASK For tasks which seem very big. enhancement tracking-issue

Comments

@dataf3l
Copy link

dataf3l commented Oct 3, 2018

This is a tracking issue for OSX support on tarpaulin

  • Figure out why my OSX tests don't have the __debug_info section
  • Update nix and libc to add ptrace support
  • Update tarpaulin to use updated nix
  • Fix other compile issues
  • Refactor ASLR disabling in OSX (not done via personality ffi call)
    • Figure out if this has worked (GDB uses same method so it should)
  • Look into the IPC ports that are used to send things like termination signals. They are necessary - ptrace on mac isn't fit for purpose
  • Investigate and update PTRACE_EVENT based code. OSX doesn't support PTRACE_EVENT
    • I've stuck in ptrace event currently but returns -1 so probably doing nothing
  • Look at how poke/peek differs with the user struct in OSX vs linux and update code if needed
  • General tidy up and refactor
  • Tracing working on ARM (for future apple devices)

Resources/links

2018/12/05 - gets into initialising the breakpoints to track coverage but none are ever hit and no exit signal.

Original issue text

localhost:src b$ rustc --version rustc 1.30.0-nightly (721913067 2018-08-26)

localhost:src b$ RUSTFLAGS="--cfg procmacro2_semver_exempt" cargo install cargo-tarpaulin
Updating registry https://github.com/rust-lang/crates.io-index
Installing cargo-tarpaulin v0.6.7
Compiling serde v1.0.78
Compiling semver-parser v0.7.0
Compiling proc-macro2 v0.4.19
Compiling version_check v0.1.4
Compiling pkg-config v0.3.14
Compiling cc v1.0.24
Compiling unicode-xid v0.1.0
Compiling num-traits v0.2.5
Compiling void v1.0.2
Compiling build_const v0.2.1
Compiling matches v0.1.8
Compiling libc v0.2.43
Compiling stable_deref_trait v1.1.1
Compiling rand_core v0.2.1
Compiling scopeguard v0.3.3
Compiling ryu v0.2.6
Compiling unicode-normalization v0.1.7
Compiling failure_derive v0.1.2
Compiling ucd-util v0.1.1
Compiling regex v1.0.5
Compiling nodrop v0.1.12
Compiling memoffset v0.2.1
Compiling percent-encoding v1.0.1
Compiling utf8-ranges v1.0.1
Compiling core-foundation-sys v0.6.1
Compiling crossbeam-utils v0.5.0
Compiling rustc-demangle v0.1.9
Compiling cfg-if v0.1.5
Compiling itoa v0.4.3
Compiling unicode-width v0.1.5
Compiling fnv v1.0.6
Compiling bitflags v1.0.4
Compiling quick-error v1.2.2
Compiling same-file v1.0.3
Compiling byteorder v1.2.6
Compiling adler32 v1.0.3
Compiling nix v0.11.0
Compiling strsim v0.7.0
Compiling termcolor v1.0.3
Compiling remove_dir_all v0.5.1
Compiling hex v0.3.2
Compiling vec_map v0.8.1
Compiling plain v0.2.3
Compiling ansi_term v0.11.0
Compiling glob v0.2.11
Compiling md5 v0.3.8
Compiling termcolor v0.3.6
Compiling shell-escape v0.1.4
Compiling fallible-iterator v0.1.5
Compiling lazycell v0.6.0
Compiling crossbeam v0.3.2
Compiling home v0.3.3
Compiling unreachable v1.0.0
Compiling unicode-bidi v0.3.4
Compiling crc v1.8.1
Compiling owning_ref v0.3.3
Compiling lazy_static v1.1.0
Compiling rand v0.5.5
Compiling memchr v2.0.2
Compiling commoncrypto-sys v0.2.0
Compiling atty v0.2.11
Compiling fs2 v0.4.3
Compiling num_cpus v1.8.0
Compiling memmap v0.6.2
Compiling arrayvec v0.4.7
Compiling regex-syntax v0.6.2
Compiling socket2 v0.3.8
Compiling log v0.4.5
Compiling filetime v0.2.1
Compiling encoding_rs v0.8.6
Compiling uuid v0.6.5
Compiling textwrap v0.10.0
Compiling humantime v1.1.1
Compiling walkdir v2.2.5
Compiling parity-wasm v0.31.3
Compiling backtrace v0.3.9
Compiling smallvec v0.6.5
Compiling cmake v0.1.33
Compiling libz-sys v1.0.20
Compiling openssl-sys v0.9.35
Compiling curl-sys v0.4.8
Compiling miniz-sys v0.1.10
Compiling lock_api v0.1.3
Compiling aho-corasick v0.6.8
Compiling commoncrypto v0.2.0
Compiling gimli v0.16.1
Compiling jobserver v0.1.11
Compiling tar v0.4.16
Compiling clap v2.32.0
Compiling tempfile v3.0.3
Compiling libssh2-sys v0.2.10
Compiling libgit2-sys v0.7.7
Compiling num-traits v0.1.43
Compiling quote v0.6.8
Compiling crypto-hash v0.3.1
Compiling core-foundation v0.6.1
Compiling thread_local v0.3.6
Compiling crossbeam-epoch v0.6.0
Compiling enum_primitive v0.1.1
Compiling semver v0.9.0
Compiling serde_json v1.0.27
Compiling toml v0.4.6
Compiling serde_ignored v0.0.4
Compiling syn v0.14.9
Compiling syn v0.15.3
Compiling gzip-header v0.1.2
Compiling flate2 v1.0.2
Compiling curl v0.4.14
Compiling rustc_version v0.2.3
Compiling idna v0.1.5
Compiling deflate v0.7.18
Compiling parking_lot_core v0.3.1
Compiling url v1.7.1
Compiling synstructure v0.9.0
Compiling scroll_derive v0.9.4
Compiling parking_lot v0.6.4
Compiling globset v0.4.2
Compiling env_logger v0.5.13
Compiling serde_derive v1.0.78
Compiling crossbeam-channel v0.2.5
Compiling scroll v0.9.0
Compiling goblin v0.0.17
Compiling ignore v0.4.4
Compiling git2 v0.7.5
Compiling failure v0.1.2
Compiling quick-xml v0.12.3
Compiling git2-curl v0.8.1
Compiling object v0.10.0
Compiling crates-io v0.17.0
Compiling coveralls-api v0.3.3
Compiling cargo v0.29.0
Compiling cargo-tarpaulin v0.6.7
error[E0432]: unresolved import nix::sys::ptrace
--> src/ptrace_control.rs:3:15
|
3 | use nix::sys::ptrace::*;
| ^^^^^^ Could not find ptrace in sys

error[E0408]: variable PTRACE_EVENT_FORK is not bound in all patterns
--> src/statemachine.rs:356:37
|
356 | PTRACE_EVENT_FORK | PTRACE_EVENT_VFORK => {
| ----------------- ^^^^^^^^^^^^^^^^^^ pattern doesn't bind PTRACE_EVENT_FORK
| |
| variable not in all patterns

error[E0408]: variable PTRACE_EVENT_VFORK is not bound in all patterns
--> src/statemachine.rs:356:17
|
356 | PTRACE_EVENT_FORK | PTRACE_EVENT_VFORK => {
| ^^^^^^^^^^^^^^^^^ ------------------ variable not in all patterns
| |
| pattern doesn't bind PTRACE_EVENT_VFORK

error[E0433]: failed to resolve. Use of undeclared type or module Options
--> src/ptrace_control.rs:13:28
|
13 | let options: Options = Options::PTRACE_O_TRACESYSGOOD |
| ^^^^^^^ Use of undeclared type or module Options

error[E0433]: failed to resolve. Use of undeclared type or module Options
--> src/ptrace_control.rs:14:9
|
14 | Options::PTRACE_O_TRACEEXEC | Options::PTRACE_O_TRACEEXIT |
| ^^^^^^^ Use of undeclared type or module Options

error[E0433]: failed to resolve. Use of undeclared type or module Options
--> src/ptrace_control.rs:14:39
|
14 | Options::PTRACE_O_TRACEEXEC | Options::PTRACE_O_TRACEEXIT |
| ^^^^^^^ Use of undeclared type or module Options

error[E0433]: failed to resolve. Use of undeclared type or module Options
--> src/ptrace_control.rs:15:9
|
15 | Options::PTRACE_O_TRACECLONE | Options::PTRACE_O_TRACEFORK |
| ^^^^^^^ Use of undeclared type or module Options

error[E0433]: failed to resolve. Use of undeclared type or module Options
--> src/ptrace_control.rs:15:40
|
15 | Options::PTRACE_O_TRACECLONE | Options::PTRACE_O_TRACEFORK |
| ^^^^^^^ Use of undeclared type or module Options

error[E0433]: failed to resolve. Use of undeclared type or module Options
--> src/ptrace_control.rs:16:9
|
16 | Options::PTRACE_O_TRACEVFORK;
| ^^^^^^^ Use of undeclared type or module Options

error[E0433]: failed to resolve. Use of undeclared type or module Request
--> src/ptrace_control.rs:31:16
|
31 | ptrace(Request::PTRACE_SINGLESTEP, pid, ptr::null_mut(), ptr::null_mut())
| ^^^^^^^ Use of undeclared type or module Request

error[E0433]: failed to resolve. Use of undeclared type or module Request
--> src/ptrace_control.rs:38:16
|
38 | ptrace(Request::PTRACE_PEEKDATA, pid, address as * mut c_void, ptr::null_mut())
| ^^^^^^^ Use of undeclared type or module Request

error[E0433]: failed to resolve. Use of undeclared type or module Request
--> src/ptrace_control.rs:47:16
|
47 | ptrace(Request::PTRACE_POKEDATA, pid, address as * mut c_void, data as * mut c_void)
| ^^^^^^^ Use of undeclared type or module Request

error[E0433]: failed to resolve. Use of undeclared type or module Request
--> src/ptrace_control.rs:54:16
|
54 | ptrace(Request::PTRACE_PEEKUSER, pid, RIP as * mut c_void, ptr::null_mut())
| ^^^^^^^ Use of undeclared type or module Request

error[E0433]: failed to resolve. Use of undeclared type or module Request
--> src/ptrace_control.rs:61:16
|
61 | ptrace(Request::PTRACE_POKEUSER, pid, RIP as * mut c_void, pc as * mut c_void)
| ^^^^^^^ Use of undeclared type or module Request

error[E0412]: cannot find type Persona in this scope
--> src/personality.rs:12:26
|
12 | const ADDR_NO_RANDOMIZE: Persona = 0x004_0000;
| ^^^^^^^ not found in this scope

error[E0412]: cannot find type Persona in this scope
--> src/personality.rs:13:20
|
13 | const GET_PERSONA: Persona = 0xFFFF_FFFF;
| ^^^^^^^ not found in this scope

error[E0412]: cannot find type Persona in this scope
--> src/personality.rs:24:25
|
24 | fn personality(persona: Persona) -> Result<c_int> {
| ^^^^^^^ not found in this scope

error[E0412]: cannot find type Options in this scope
--> src/ptrace_control.rs:13:18
|
13 | let options: Options = Options::PTRACE_O_TRACESYSGOOD |
| ^^^^^^^ did you mean Option?

error[E0425]: cannot find function setoptions in this scope
--> src/ptrace_control.rs:17:5
|
17 | setoptions(pid, options)
| ^^^^^^^^^^ did you mean options?

error[E0425]: cannot find function detach in this scope
--> src/ptrace_control.rs:21:5
|
21 | detach(pid)
| ^^^^^^ not found in this scope

error[E0425]: cannot find function cont in this scope
--> src/ptrace_control.rs:25:5
|
25 | cont(pid, sig)
| ^^^^ not found in this scope

error[E0425]: cannot find function ptrace in this scope
--> src/ptrace_control.rs:31:9
|
31 | ptrace(Request::PTRACE_SINGLESTEP, pid, ptr::null_mut(), ptr::null_mut())
| ^^^^^^ not found in this scope
help: possible candidate is found in another module, you can import it into scope
|
1 | use nix::libc::ptrace;
|

error[E0425]: cannot find function ptrace in this scope
--> src/ptrace_control.rs:38:9
|
38 | ptrace(Request::PTRACE_PEEKDATA, pid, address as * mut c_void, ptr::null_mut())
| ^^^^^^ not found in this scope
help: possible candidate is found in another module, you can import it into scope
|
1 | use nix::libc::ptrace;
|

error[E0425]: cannot find function ptrace in this scope
--> src/ptrace_control.rs:47:9
|
47 | ptrace(Request::PTRACE_POKEDATA, pid, address as * mut c_void, data as * mut c_void)
| ^^^^^^ not found in this scope
help: possible candidate is found in another module, you can import it into scope
|
1 | use nix::libc::ptrace;
|

error[E0425]: cannot find function ptrace in this scope
--> src/ptrace_control.rs:54:9
|
54 | ptrace(Request::PTRACE_PEEKUSER, pid, RIP as * mut c_void, ptr::null_mut())
| ^^^^^^ not found in this scope
help: possible candidate is found in another module, you can import it into scope
|
1 | use nix::libc::ptrace;
|

error[E0425]: cannot find function ptrace in this scope
--> src/ptrace_control.rs:61:9
|
61 | ptrace(Request::PTRACE_POKEUSER, pid, RIP as * mut c_void, pc as * mut c_void)
| ^^^^^^ not found in this scope
help: possible candidate is found in another module, you can import it into scope
|
1 | use nix::libc::ptrace;
|

error[E0425]: cannot find function traceme in this scope
--> src/ptrace_control.rs:66:5
|
66 | traceme()
| ^^^^^^^ not found in this scope

error[E0425]: cannot find function getevent in this scope
--> src/ptrace_control.rs:70:5
|
70 | getevent(pid)
| ^^^^^^^^ not found in this scope

error[E0599]: no associated item named __WALL found for type nix::sys::wait::WaitPidFlag in the current scope
--> src/statemachine.rs:231:75
|
231 | let wait = waitpid(Pid::from_raw(-1), Some(WaitPidFlag::WNOHANG | WaitPidFlag::__WALL));
| ^^^^^^^^^^^^^^^^^^^ associated item not found in nix::sys::wait::WaitPidFlag

error[E0599]: no variant named PtraceEvent found for type nix::sys::wait::WaitStatus in the current scope
--> src/statemachine.rs:251:13
|
251 | WaitStatus::PtraceEvent(c,s,e) => {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ variant not found in nix::sys::wait::WaitStatus

error: aborting due to 30 previous errors

Some errors occurred: E0408, E0412, E0425, E0432, E0433, E0599.
For more information about an error, try rustc --explain E0408.
error: failed to compile cargo-tarpaulin v0.6.7, intermediate artifacts can be found at /var/folders/26/ghqp1zl50yq3bp47zljpksk80000gn/T/cargo-install5pqpq9

Caused by:
Could not compile cargo-tarpaulin.

To learn more, run the command again with --verbose.

@xd009642
Copy link
Owner

xd009642 commented Oct 3, 2018

So tarpaulin currently only supports linux, if anything else works it's more luck that intent. OSX compared to linux doesn't have a complete ptrace implementation so the nix crate doesn't expose the ptrace bindings.

If anyone was interested in adding OSX support they'd need to either replace ptrace with appropriate mac system calls or contribute to nix to add the parts of ptrace mac exports and for the rest of it add the appropriate system calls.

I don't have any Macs so while I'm open to more OS support this is something I'd appreciate help on.

@dataf3l
Copy link
Author

dataf3l commented Oct 3, 2018

Do you know of any tools that can be used on OSX for coverage reports?,
Also, I have a mac and can help test whatever you guys code, but I'm not smart enough to do the syscalls on OSX, I need to know more about that, first.
I can also provide ssh access I guess, via Hamachi, if somebody wants to to connect to my machine and work remotely on it, but I assume that wouldn't work out, since the people who are experts on osx syscalls and stuff like that probably already have and love a mac.

would a donation to the project help?

@xd009642
Copy link
Owner

xd009642 commented Oct 4, 2018

You might be able to get it working with docker on mac - though I'm not really that familiar with docker so maybe not... A good start if you wanted to help for the ptrace calls on mac. It should just be a case of looking at the ptrace docs for OSX finding what's supported and changing the cfgs in https://github.com/nix-rust/nix/ to match. Once that's done and merged in it we can see what fails to compile and go from there.

As for donations I hadn't considered project donations before, though I'm probably going to be rather busy with my day job till the end of the year so that probably won't change anything in reality.

@xd009642
Copy link
Owner

xd009642 commented Oct 7, 2018

Oh realised I missed your first question on tools that can be used for coverage, there is https://github.com/SimonKagstrom/kcov which works on linux, freebsd and OSX.

I've just opened a PR on rust-nix/nix nix-rust/nix#949 for ptrace support in OSX and BSD. So once that's done, released and I update tarpaulin for the new nix version tarpaulin might work on OSX or there might be other things that need adapting. I'll ping you when there' something to test though!

Edit: Notes for me -

@xd009642
Copy link
Owner

xd009642 commented Oct 8, 2018

@dataf3l could you clone https://github.com/xd009642/nix/ checkout the branch issue-947 and let me know the output of the following tests sys::test_ptrace::test_ptrace sys::test_ptrace::test_ptrace_cont? This is part of adding OSX support to some of my dependencies and the travis runs are failing so we're trying to figure out if it's a travis qemu issue or an actual apple issue

@dataf3l
Copy link
Author

dataf3l commented Oct 8, 2018

@xd009642 Unfortunately, it looks like it some of the tests didn't pass...

Build output

``` localhost:~ b$ cd work/ localhost:work b$ git clone https://github.com/xd009642/nix.git -b issue-947 Cloning into 'nix'... remote: Enumerating objects: 16, done. remote: Counting objects: 100% (16/16), done. remote: Compressing objects: 100% (13/13), done. remote: Total 7722 (delta 4), reused 6 (delta 3), pack-reused 7706 Receiving objects: 100% (7722/7722), 2.02 MiB | 990.00 KiB/s, done. Resolving deltas: 100% (5583/5583), done. localhost:work b$ cd nix/ localhost:nix b$ ls bors.toml build.rs Cargo.toml CHANGELOG.md ci CONTRIBUTING.md CONVENTIONS.md Cross.toml LICENSE README.md RELEASE_PROCEDURE.md src test localhost:nix b$ git status . On branch issue-947 Your branch is up to date with 'origin/issue-947'.

nothing to commit, working tree clean
localhost:nix b$ cargo build
Updating registry https://github.com/rust-lang/crates.io-index
Updating git repository https://github.com/rust-lang/libc
Compiling libc v0.2.43 (https://github.com/rust-lang/libc#41944d5c)
Compiling nix v0.11.0 (file:///Users/b/work/nix)
Compiling cfg-if v0.1.5
Compiling void v1.0.2
Compiling bitflags v1.0.4
Finished dev [unoptimized + debuginfo] target(s) in 23.98s
localhost:nix b$ cargo test
Downloading rand_core v0.2.2
Downloading rand_core v0.3.0
Compiling rand_core v0.3.0
Compiling version_check v0.1.5
Compiling libc v0.2.43
Compiling remove_dir_all v0.5.1
Compiling byteorder v1.2.6
Compiling rand_core v0.2.2
Compiling lazy_static v1.1.0
Compiling iovec v0.1.2
Compiling rand v0.5.5
Compiling bytes v0.4.10
Compiling tempfile v3.0.4
Compiling nix v0.11.0 (file:///Users/b/work/nix)
warning: unused macro definition
--> test/test_mount.rs:209:1
|
209 | / macro_rules! run_tests {
210 | | ( $($test_fn:ident),* ) => {{
211 | | println!();
212 | |
... |
220 | | }}
221 | | }
| |_^
|
= note: #[warn(unused_macros)] on by default

warning: unused macro definition
--> test/test.rs:12:1
|
12 | / macro_rules! skip_if_not_root {
13 | | ($name:expr) => {
14 | | use nix::unistd::Uid;
15 | | use std;
... |
24 | | };
25 | | }
| |_^
|
= note: #[warn(unused_macros)] on by default

warning: unused import: std::mem
--> test/sys/test_ptrace.rs:8:5
|
8 | use std::mem;
| ^^^^^^^^
|
= note: #[warn(unused_imports)] on by default

warning: unused import: nix::sys::signal::SigSet
--> test/test_poll.rs:2:5
|
2 | use nix::sys::signal::SigSet;
| ^^^^^^^^^^^^^^^^^^^^^^^^

warning: unused imports: TimeSpec, TimeValLike
--> test/test_poll.rs:3:22
|
3 | use nix::sys::time::{TimeSpec, TimeValLike};
| ^^^^^^^^ ^^^^^^^^^^^

warning: use of deprecated item 'nix::unistd::pipe2': pipe2(2) is not actually atomic on these platforms. Use pipe(2) and fcntl(2) instead
--> test/test_unistd.rs:386:22
|
386 | let (fd0, fd1) = pipe2(OFlag::O_CLOEXEC).unwrap();
| ^^^^^
|
= note: #[warn(deprecated)] on by default

warning: unused std::result::Result which must be used
--> test/test_stat.rs:177:5
|
177 | utimes(&fullpath, &TimeVal::seconds(9990), &TimeVal::seconds(5550));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: #[warn(unused_must_use)] on by default
= note: this Result may be an Err variant, which should be handled

Finished dev [unoptimized + debuginfo] target(s) in 13.20s
 Running target/debug/deps/nix-f5796617f4709c01

running 35 tests
test sys::event::test_struct_kevent ... ok
test ifaddrs::tests::test_getifaddrs ... ok
test sys::select::tests::fdset_clear ... ok
test sys::select::tests::fdset_insert ... ok
test sys::select::tests::fdset_remove ... ok
test sys::select::tests::test_select ... ok
test sys::select::tests::test_select_nfds ... ok
test sys::select::tests::test_select_nfds2 ... ok
test sys::signal::tests::test_clear ... ok
test sys::select::tests::fdset_highest ... ok
test sys::signal::tests::test_contains ... ok
test sys::signal::tests::test_extend ... ok
test sys::signal::tests::test_from_str_invalid_value ... ok
test sys::signal::tests::test_sigaction ... ok
test sys::signal::tests::test_thread_signal_block ... ok
test sys::signal::tests::test_thread_signal_get_mask ... ok
test sys::signal::tests::test_from_str_round_trips ... ok
test sys::signal::tests::test_thread_signal_set_mask ... ok
test sys::signal::tests::test_thread_signal_swap ... ok
test sys::signal::tests::test_thread_signal_unblock ... ok
test sys::socket::addr::tests::test_macos_loopback_datalink_addr ... ok
test sys::socket::addr::tests::test_macos_tap_datalink_addr ... ok
test sys::socket::sockopt::test::is_socket_type_dgram ... ok
test sys::statvfs::test::fstatvfs_call ... ok
test sys::statvfs::test::statvfs_call ... ok
test sys::socket::sockopt::test::is_socket_type_unix ... ok
test sys::time::test::test_timespec ... ok
test sys::time::test::test_timespec_neg ... ok
test sys::time::test::test_timespec_fmt ... ok
test sys::time::test::test_timespec_ord ... ok
test sys::time::test::test_timeval ... ok
test sys::time::test::test_timeval_neg ... ok
test sys::time::test::test_timeval_fmt ... ok
test sys::time::test::test_timeval_ord ... ok
test sys::utsname::test::test_uname_darwin ... ok

test result: ok. 35 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

 Running target/debug/deps/test-fc2b5d0af9911a3b

running 98 tests
test sys::test_aio::test_accessors ... ok
test sys::test_aio::test_aio_suspend ... ok
test sys::test_aio::test_read_bytes_mut_small ... ok
test sys::test_aio::test_aio_cancel_all ... ok
test sys::test_aio::test_fsync_error ... ok
test sys::test_aio::test_read_error ... ok
test sys::test_aio::test_read_immutable_buffer ... ok
test sys::test_aio::test_write_error ... ok
test sys::test_aio::test_fsync ... ok
test sys::test_aio::test_cancel ... ok
test sys::test_ioctl::bsd::test_op_none ... ok
test sys::test_ioctl::bsd::test_op_read ... ok
test sys::test_ioctl::bsd::test_op_read_64 ... ok
test sys::test_ioctl::bsd::test_op_read_write ... ok
test sys::test_ioctl::bsd::test_op_read_write_64 ... ok
test sys::test_ioctl::bsd::test_op_write ... ok
test sys::test_ioctl::bsd::test_op_write_64 ... ok
test sys::test_pthread::test_pthread_self ... ok
test sys::test_aio::test_read_into_mut_slice ... ok
test sys::test_aio::test_read ... ok
test sys::test_ptrace::test_ptrace ... FAILED
test sys::test_aio::test_write_bytes ... ok
test sys::test_aio::test_read_into_pointer ... ok
test sys::test_aio::test_write_from_pointer ... ok
test sys::test_aio::test_write ... ok
test sys::test_select::test_pselect_nfds2 ... ok
test sys::test_ptrace::test_ptrace_cont ... FAILED
test sys::test_signal::test_kill_none ... ok
test sys::test_signal::test_old_sigaction_flags ... ok
test sys::test_signal::test_sigprocmask_noop ... ok
test sys::test_socket::test_inetv4_addr_to_sock_addr ... ok
test sys::test_socket::test_inetv6_addr_to_sock_addr ... ok
test sys::test_socket::test_getsockname ... ok
test sys::test_socket::test_path_to_sock_addr ... ok
test sys::test_socket::test_scm_rights ... ok
test sys::test_socket::test_socketpair ... ok
test sys::test_socket::test_scm_rights_single_cmsg_multiple_fds ... ok
test sys::test_socket::test_sendmsg_empty_cmsgs ... ok
test sys::test_socket::test_syscontrol ... ok
test sys::test_termios::test_cfmakeraw ... ok
test sys::test_termios::test_tcgetattr_ebadf ... ok
test sys::test_termios::test_local_flags ... ok
test sys::test_sockopt::test_so_buf ... ok
test sys::test_termios::test_output_flags ... ok
test sys::test_socket::test_unixdomain ... ok
test sys::test_aio::test_write_sigev_signal ... ok
test sys::test_select::test_pselect ... ok
test sys::test_signal::test_sigprocmask ... ok
test sys::test_termios::test_tcgetattr_pty ... ok
test sys::test_termios::test_tcgetattr_enotty ... ok
test sys::test_uio::test_pwrite ... ok
test sys::test_wait::test_waitstatus_from_raw ... ok
test sys::test_uio::test_readv ... ok
test test_dir::ebadf ... ok
test sys::test_wait::test_wait_exit ... ok
test sys::test_uio::test_pread ... ok
test sys::test_wait::test_wait_signal ... ok
test sys::test_uio::test_writev ... ok
test sys::test_wait::test_waitstatus_pid ... ok
test test_dir::rewind ... ok
test test_net::test_if_nametoindex ... ok
test test_poll::test_poll_debug ... ok
test test_dir::read ... ok
test test_fcntl::test_openat ... ok
test test_pty::test_open_ptty_pair ... ok
test test_pty::test_openpty ... ok
test test_fcntl::test_readlink ... ok
test test_pty::test_openpty_with_termios ... ok
test test_pty::test_explicit_close ... ok
test test_sendfile::test_sendfile_darwin ... ok
test test_stat::test_fchmod ... ok
test test_stat::test_fchmodat ... ok
test test_stat::test_fstatat ... ok
test test_stat::test_futimens ... ok
test test_stat::test_stat_and_fstat ... ok
test test_stat::test_utimes ... ok
test test_stat::test_utimensat ... ok
test test_stat::test_stat_fstat_lstat ... ok
test test_unistd::test_fchdir ... ok
test test_unistd::test_fpathconf_limited ... ok
test test_unistd::test_getpid ... ok
test test_unistd::test_getsid ... ok
test test_unistd::test_lseek ... ok
test test_unistd::test_mkfifo_directory ... ok
test test_unistd::test_mkfifo ... ok
test test_unistd::test_mkstemp_directory ... ok
test test_unistd::test_mkstemp ... ok
test test_unistd::test_pathconf_limited ... ok
test test_unistd::test_pipe ... ok
test test_unistd::test_pipe2 ... ok
test test_unistd::test_sysconf_limited ... ok
test test_unistd::test_getcwd ... ok
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
test test_unistd::test_execve ... ok
test test_unistd::test_fork_and_waitpid ... ok
test test_unistd::test_wait ... ok
test test_poll::test_poll ... ok
test test_unistd::test_alarm ... ok
test test_unistd::test_canceling_alarm ... ok

failures:

---- sys::test_ptrace::test_ptrace stdout ----
thread 'sys::test_ptrace::test_ptrace' panicked at 'assertion failed: err == Error::Sys(Errno::EPERM) || err == Error::Sys(Errno::ENOSYS)', test/sys/test_ptrace.rs:15:5

---- sys::test_ptrace::test_ptrace_cont stdout ----
thread 'sys::test_ptrace::test_ptrace_cont' panicked at 'called Result::unwrap() on an Err value: Sys(ENOTSUP)', libcore/result.rs:983:5

failures:
sys::test_ptrace::test_ptrace
sys::test_ptrace::test_ptrace_cont

test result: FAILED. 96 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out

error: test failed, to rerun pass '--test test'
localhost:nix b$

</p>
</details>

@xd009642
Copy link
Owner

xd009642 commented Oct 8, 2018

Okay in src/sys/ptrace/bsd.rs in the function attach could you replace Request::PT_ATTACH with Request::PT_ATTACHEXC? Okay never mind I expect that to fail.

@dataf3l
Copy link
Author

dataf3l commented Oct 9, 2018

Still no dice:

Build output

localhost:nix b$ vi src/sys/ptrace/bsd.rs localhost:nix b$ cargo clean localhost:nix b$ cargo build Compiling libc v0.2.43 (https://github.com/rust-lang/libc#41944d5c) Compiling nix v0.11.0 (file:///Users/b/work/nix) Compiling bitflags v1.0.4 Compiling cfg-if v0.1.5 Compiling void v1.0.2 Finished dev [unoptimized + debuginfo] target(s) in 8.92s localhost:nix b$ localhost:nix b$ localhost:nix b$ brew install sphinx-doc Updating Homebrew... ==> Auto-updated Homebrew! Updated 2 taps (homebrew/core and homebrew/cask). ==> Updated Formulae aws-elasticbeanstalk futhark jhipster php-code-sniffer swiftgen terragrunt

==> Downloading https://homebrew.bintray.com/bottles/sphinx-doc-1.8.1.high_sierra.bottle.tar.gz
######################################################################## 100.0%
==> Pouring sphinx-doc-1.8.1.high_sierra.bottle.tar.gz
==> Caveats
sphinx-doc is keg-only, which means it was not symlinked into /usr/local,
because this formula is mainly used internally by other formulae.
Users are advised to use pip to install sphinx-doc.

If you need to have sphinx-doc first in your PATH run:
echo 'export PATH="/usr/local/opt/sphinx-doc/bin:$PATH"' >> ~/.bash_profile

==> Summary
🍺 /usr/local/Cellar/sphinx-doc/1.8.1: 2,695 files, 46.3MB
localhost:nix b$ cargo test
Compiling version_check v0.1.5
Compiling rand_core v0.3.0
Compiling libc v0.2.43
Compiling byteorder v1.2.6
Compiling remove_dir_all v0.5.1
Compiling rand_core v0.2.2
Compiling lazy_static v1.1.0
Compiling rand v0.5.5
Compiling iovec v0.1.2
Compiling bytes v0.4.10
Compiling tempfile v3.0.4
Compiling nix v0.11.0 (file:///Users/b/work/nix)
warning: unused macro definition
--> test/test_mount.rs:209:1
|
209 | / macro_rules! run_tests {
210 | | ( $($test_fn:ident),* ) => {{
211 | | println!();
212 | |
... |
220 | | }}
221 | | }
| |_^
|
= note: #[warn(unused_macros)] on by default

warning: unused macro definition
--> test/test.rs:12:1
|
12 | / macro_rules! skip_if_not_root {
13 | | ($name:expr) => {
14 | | use nix::unistd::Uid;
15 | | use std;
... |
24 | | };
25 | | }
| |_^
|
= note: #[warn(unused_macros)] on by default

warning: unused import: std::mem
--> test/sys/test_ptrace.rs:8:5
|
8 | use std::mem;
| ^^^^^^^^
|
= note: #[warn(unused_imports)] on by default

warning: unused import: nix::sys::signal::SigSet
--> test/test_poll.rs:2:5
|
2 | use nix::sys::signal::SigSet;
| ^^^^^^^^^^^^^^^^^^^^^^^^

warning: unused imports: TimeSpec, TimeValLike
--> test/test_poll.rs:3:22
|
3 | use nix::sys::time::{TimeSpec, TimeValLike};
| ^^^^^^^^ ^^^^^^^^^^^

warning: use of deprecated item 'nix::unistd::pipe2': pipe2(2) is not actually atomic on these platforms. Use pipe(2) and fcntl(2) instead
--> test/test_unistd.rs:386:22
|
386 | let (fd0, fd1) = pipe2(OFlag::O_CLOEXEC).unwrap();
| ^^^^^
|
= note: #[warn(deprecated)] on by default

warning: unused std::result::Result which must be used
--> test/test_stat.rs:177:5
|
177 | utimes(&fullpath, &TimeVal::seconds(9990), &TimeVal::seconds(5550));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: #[warn(unused_must_use)] on by default
= note: this Result may be an Err variant, which should be handled

Finished dev [unoptimized + debuginfo] target(s) in 10.49s
 Running target/debug/deps/nix-f5796617f4709c01

running 35 tests
test sys::event::test_struct_kevent ... ok
test sys::select::tests::fdset_insert ... ok
test sys::select::tests::fdset_clear ... ok
test sys::select::tests::test_select ... ok
test sys::select::tests::test_select_nfds2 ... ok
test ifaddrs::tests::test_getifaddrs ... ok
test sys::select::tests::test_select_nfds ... ok
test sys::select::tests::fdset_remove ... ok
test sys::signal::tests::test_clear ... ok
test sys::signal::tests::test_contains ... ok
test sys::signal::tests::test_extend ... ok
test sys::signal::tests::test_from_str_invalid_value ... ok
test sys::select::tests::fdset_highest ... ok
test sys::signal::tests::test_sigaction ... ok
test sys::signal::tests::test_thread_signal_block ... ok
test sys::signal::tests::test_thread_signal_get_mask ... ok
test sys::signal::tests::test_from_str_round_trips ... ok
test sys::signal::tests::test_thread_signal_set_mask ... ok
test sys::signal::tests::test_thread_signal_swap ... ok
test sys::signal::tests::test_thread_signal_unblock ... ok
test sys::socket::addr::tests::test_macos_loopback_datalink_addr ... ok
test sys::socket::addr::tests::test_macos_tap_datalink_addr ... ok
test sys::socket::sockopt::test::is_socket_type_dgram ... ok
test sys::statvfs::test::fstatvfs_call ... ok
test sys::statvfs::test::statvfs_call ... ok
test sys::time::test::test_timespec ... ok
test sys::time::test::test_timespec_neg ... ok
test sys::time::test::test_timespec_ord ... ok
test sys::time::test::test_timespec_fmt ... ok
test sys::socket::sockopt::test::is_socket_type_unix ... ok
test sys::time::test::test_timeval ... ok
test sys::time::test::test_timeval_fmt ... ok
test sys::time::test::test_timeval_neg ... ok
test sys::time::test::test_timeval_ord ... ok
test sys::utsname::test::test_uname_darwin ... ok

test result: ok. 35 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

 Running target/debug/deps/test-fc2b5d0af9911a3b

running 98 tests
test sys::test_aio::test_accessors ... ok
test sys::test_aio::test_aio_suspend ... ok
test sys::test_aio::test_fsync_error ... ok
test sys::test_aio::test_read_error ... ok
test sys::test_aio::test_read_immutable_buffer ... ok
test sys::test_aio::test_cancel ... ok
test sys::test_aio::test_aio_cancel_all ... ok
test sys::test_aio::test_read_bytes_mut_small ... ok
test sys::test_aio::test_read ... ok
test sys::test_aio::test_fsync ... ok
test sys::test_aio::test_write_error ... ok
test sys::test_ioctl::bsd::test_op_none ... ok
test sys::test_ioctl::bsd::test_op_read ... ok
test sys::test_aio::test_read_into_mut_slice ... ok
test sys::test_aio::test_read_into_pointer ... ok
test sys::test_ioctl::bsd::test_op_read_64 ... ok
test sys::test_ioctl::bsd::test_op_read_write ... ok
test sys::test_ioctl::bsd::test_op_read_write_64 ... ok
test sys::test_ioctl::bsd::test_op_write ... ok
test sys::test_aio::test_write ... ok
test sys::test_ioctl::bsd::test_op_write_64 ... ok
test sys::test_pthread::test_pthread_self ... ok
test sys::test_ptrace::test_ptrace ... FAILED
test sys::test_signal::test_old_sigaction_flags ... ok
test sys::test_select::test_pselect_nfds2 ... ok
test sys::test_signal::test_kill_none ... ok
test sys::test_signal::test_sigprocmask_noop ... ok
test sys::test_ptrace::test_ptrace_cont ... FAILED
test sys::test_socket::test_inetv4_addr_to_sock_addr ... ok
test sys::test_socket::test_inetv6_addr_to_sock_addr ... ok
test sys::test_socket::test_path_to_sock_addr ... ok
test sys::test_socket::test_scm_rights ... ok
test sys::test_socket::test_scm_rights_single_cmsg_multiple_fds ... ok
test sys::test_socket::test_sendmsg_empty_cmsgs ... ok
test sys::test_socket::test_socketpair ... ok
test sys::test_socket::test_syscontrol ... ok
test sys::test_socket::test_getsockname ... ok
test sys::test_termios::test_cfmakeraw ... ok
test sys::test_sockopt::test_so_buf ... ok
test sys::test_termios::test_local_flags ... ok
test sys::test_termios::test_tcgetattr_ebadf ... ok
test sys::test_socket::test_unixdomain ... ok
test sys::test_termios::test_output_flags ... ok
test sys::test_termios::test_tcgetattr_pty ... ok
test sys::test_termios::test_tcgetattr_enotty ... ok
test sys::test_uio::test_pwrite ... ok
test sys::test_uio::test_pread ... ok
test sys::test_uio::test_readv ... ok
test sys::test_wait::test_wait_exit ... ok
test sys::test_wait::test_waitstatus_from_raw ... ok
test sys::test_wait::test_wait_signal ... ok
test sys::test_uio::test_writev ... ok
test test_dir::ebadf ... ok
test sys::test_aio::test_write_bytes ... ok
test sys::test_aio::test_write_from_pointer ... ok
test sys::test_wait::test_waitstatus_pid ... ok
test test_net::test_if_nametoindex ... ok
test sys::test_aio::test_write_sigev_signal ... ok
test sys::test_select::test_pselect ... ok
test test_dir::rewind ... ok
test test_fcntl::test_openat ... ok
test sys::test_signal::test_sigprocmask ... ok
test test_poll::test_poll_debug ... ok
test test_pty::test_open_ptty_pair ... ok
test test_fcntl::test_readlink ... ok
test test_pty::test_openpty ... ok
test test_dir::read ... ok
test test_pty::test_openpty_with_termios ... ok
test test_pty::test_explicit_close ... ok
test test_sendfile::test_sendfile_darwin ... ok
test test_stat::test_fchmodat ... ok
test test_stat::test_fstatat ... ok
test test_stat::test_fchmod ... ok
test test_stat::test_stat_and_fstat ... ok
test test_stat::test_futimens ... ok
test test_stat::test_stat_fstat_lstat ... ok
test test_stat::test_utimes ... ok
test test_stat::test_utimensat ... ok
test test_unistd::test_fchdir ... ok
test test_unistd::test_getpid ... ok
test test_unistd::test_fpathconf_limited ... ok
test test_unistd::test_getsid ... ok
test test_unistd::test_lseek ... ok
test test_unistd::test_mkfifo_directory ... ok
test test_unistd::test_mkfifo ... ok
test test_unistd::test_mkstemp_directory ... ok
test test_unistd::test_mkstemp ... ok
test test_unistd::test_pathconf_limited ... ok
test test_unistd::test_pipe ... ok
test test_unistd::test_pipe2 ... ok
test test_unistd::test_sysconf_limited ... ok
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
test test_unistd::test_getcwd ... ok
test test_unistd::test_execve ... ok
test test_unistd::test_fork_and_waitpid ... ok
test test_unistd::test_wait ... ok
test test_poll::test_poll ... ok
test test_unistd::test_alarm ... ok
test test_unistd::test_canceling_alarm ... ok

failures:

---- sys::test_ptrace::test_ptrace stdout ----
thread 'sys::test_ptrace::test_ptrace' panicked at 'assertion failed: err == Error::Sys(Errno::EPERM) || err == Error::Sys(Errno::ENOSYS)', test/sys/test_ptrace.rs:15:5

---- sys::test_ptrace::test_ptrace_cont stdout ----
thread 'sys::test_ptrace::test_ptrace_cont' panicked at 'called Result::unwrap() on an Err value: Sys(ENOTSUP)', libcore/result.rs:983:5

failures:
sys::test_ptrace::test_ptrace
sys::test_ptrace::test_ptrace_cont

test result: FAILED. 96 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out

error: test failed, to rerun pass '--test test'
localhost:nix b$ git diff
diff --git a/src/sys/ptrace/bsd.rs b/src/sys/ptrace/bsd.rs
index ee9b6eb..45c8fa2 100644
--- a/src/sys/ptrace/bsd.rs
+++ b/src/sys/ptrace/bsd.rs
@@ -77,7 +77,7 @@ pub fn traceme() -> Result<()> {
///
/// Attaches to the process specified in pid, making it a tracee of the calling process.
pub fn attach(pid: Pid) -> Result<()> {

  • unsafe { ptrace_other(Request::PT_ATTACH, pid, ptr::null_mut(), 0).map(|_| ()) }
  • unsafe { ptrace_other(Request::PT_ATTACHEXC, pid, ptr::null_mut(), 0).map(|_| ()) }
    }

/// Detaches the current running process, as with ptrace(PT_DETACH, ...)
localhost:nix b$

@dataf3l
Copy link
Author

dataf3l commented Oct 9, 2018

I guess I could setup a cron job to clone the repo and run the tests on this machine, like daily, if that is something you guys would find useful.

@xd009642
Copy link
Owner

xd009642 commented Oct 9, 2018

No that's alright I got it more working some point last night and affirmed that travis is fine for testing. Thanks for offering though! Once this is in and released I'll be able to apply some changes to tarpaulin and may ask you to test them

@dataf3l
Copy link
Author

dataf3l commented Oct 9, 2018 via email

@xd009642
Copy link
Owner

There's now a tracking branch for this. https://github.com/xd009642/tarpaulin/tree/osx-support I don't yet expect it to work but the ptrace compiler errors should be gone (in theory)

@xd009642 xd009642 changed the title It doesn't build in OSX OSX support tracking issue Oct 23, 2018
@xd009642
Copy link
Owner

So tarpaulin now builds but doesn't work on OSX. I'm going to edit your first comment to add a checklist of things that need doing so it actually shows up towards the issue progress and to generally keep things tidy. Gonna try and keep the original text in a collapsable markdown block for posterity though

@dataf3l
Copy link
Author

dataf3l commented Oct 24, 2018

I like it because it makes my comment look more professional.
Screw the past, bring on the future! :)

@xd009642
Copy link
Owner

xd009642 commented Dec 5, 2018

So as an update since I've been working on this issue this week. I appear to have ported over most of the ptrace stuff launch a test add the breakpoints to instrument it but after initialising the test it times out.

This makes me think the trace isn't being passed down to the child threads and rustc_test does spawn a thread for each test so it can kill it if it runs too long. Hopefully, once I solve this issue I might see some coverage results come out (if I'm really lucky and everything else works to some degree).

@xd009642
Copy link
Owner

xd009642 commented Dec 7, 2018

@dataf3l it would be helpful if you could take the latest build off the osx-support branch and create the default lib project cargo new osxtest --lib run tarpaulin with verbose and then use ddbug on the test binary it attempts to process. After Launching test in the tarpaulin verbose printout it prints the path to the test binary.

Also, if the outputs big upload it in a text file or link to a gist

@andrewtj
Copy link

@xd009642 I've just wandered in from "This Week in Rust" and have zero context for this issue, but I gave it a go:

$ ddbug osxtest-21111c0e5489441f
ERROR 2018-12-13T00:24:24Z: ddbug: osxtest-21111c0e5489441f: unable to locate debug file: dSYM not found

osxtest-21111c0e5489441f.dSYM exists in that folder for whatever it's worth.

@xd009642
Copy link
Owner

Oh that's actually super useful! Could you run ddbug on the dsym file?

@xd009642
Copy link
Owner

Awesome that's what I expected to see! I've been trying to debug and develop this via travis and apples online docs which has been a bit of a mare. I've pushed a change to the osx-support branch and if results come out we should have the first beta of OSX support.

If more work is needed (I have a feeling there is more) and you're interested in working on the tarpaulin code I've got a developer wiki that explains some of how the project works https://github.com/xd009642/tarpaulin/wiki/Developers if there are any missing parts or issues I do also have a tracking issue for the wiki since it's relatively new 👍

@andrewtj
Copy link

That overview's given me the gist of how this is supposed to work 👍. At the minute osx-support fails at the mmap call because the path it's provided is a bundle (folder). With the change below to test_loader::open_symbols_file the mmap call succeeds:

 #[cfg(target_os = "macos")]
 fn open_symbols_file(test: &Path) -> io::Result<File> {
-    let d_sym = test.with_extension("dSYM");
+    let mut d_sym = test.with_extension("dSYM");
+    d_sym.push("Contents/Resources/DWARF");
+    d_sym.push(test.file_stem().unwrap());
     println!("Test file {}", d_sym.display());
     File::open(&d_sym)
 }

Which leads to:

[...]
Processing compilation units
Getting the entry points
Processed all compilation units
trace children -1
Failed to trace child threads
Number of traces 6
Finished instrumenting code

I guess this could be a privilege issue (codesigning/entitlements) or some macOS API deviation. I've got to shelf this for the moment but I'll try and circle back to this in the next few days.

@xd009642
Copy link
Owner

So the mach-o loading side of thing looks like it's working with that change as it's found the traces from the DWARF tables. This is in the update loop actually attempting to trace. I think this is a case of src/process_handling/bsd.rs the trace_children function being incorrect as ptrace isn't tracing spawned threads.

Apple doesn't expose the constants but provides more ptrace functionality than they advertise so I hard coded it myself to see if it would work. So looks like back to the apple ptrace drawing board for that

@xd009642
Copy link
Owner

@nathanwhit sorry for the random tag but I saw you tried some stuff with OSX support in a fork, did you make any progress or have any insights? I'm going to be borrowing a mac from a friend this week so might be able to make some real progress once I can run the code locally

@nathanwhit
Copy link

nathanwhit commented Feb 23, 2020

@xd009642 No worries! I did make some progress. The main issue is that (as you noticed) ptrace on macOS doesn't have the right capabilities so you have to work through other means. More specifically you have to go through the kernel's mach interface. I have two branches with my progress, I'll go through the progress and problems with each.

My first attempt was the macos branch, which implements most of the functionality required in the ptrace_control and process_handling modules. If you run one of simple coverage tests it usually passes, but not always, which leads into the problem. I didn't modify the state machine much, and so it performs most actions using the pid to identify threads. In the mach interface, however, you don't operate on things by pid, instead you work on mach tasks and mach threads which are a different abstraction. For instance, to modify the instruction pointer or write to memory as in the write_to_address fn you operate on a mach thread, not a Unix/POSIX thread. There's a mapping from pid to the mach task responsible for it, which then contains info about the mach threads in the task. However, there's no mapping from pid to mach thread. So currently functions like write_to_address take a pid, lookup the mach task associated with it, then have no way of knowing which mach thread the pid refers to, so it just chooses the first thread it finds (which is wrong a good portion of the time).
In essence the problem is that the state machine and actual operations operate on different abstractions and there's not a clean way to map between them (AFAIK).

My second attempt was the macos-v2 branch. In this I wrote up wrappers/abstractions for the relevant mach structures, and figured out how to receive the signals we would usually get from waitpid in a mach message form. This way the state machine could operate on the mach structures instead of on the Unix/POSIX architecture. The basic structure is there, and the mach messages are properly received. The problem is I didn't update the state machine to use the new structures, and was becoming frustrated with working on the mach end of things (as it lacks documentation badly and is fairly opaque) which was demotivating. As it stands then, this branch doesn't pass any tests and doesn't run properly due to not adjusting the state machines. I think this branch has more potential though, as it seems like a more "correct" approach.

I've pushed my latest changes to both branches, you're welcome to continue where I left off.

@nathanwhit
Copy link

One important note is that for any of it to work you have to run with sudo (i.e. sudo cargo test), as macOS requires elevated privileges for the useful mach interfaces.

Also, if you want to work/mess around with just the mach side of things I have a repo with some basic tests I used for debugging/developing the mach interface. Just build a binary with the name a.out and sudo cargo run. As it is, it just prints out the signals received and some debugging info about the mach task/threads. It may or may not be useful for you, but it was definitely helpful for me to reason about the mach side without worrying too much about tarpaulin's specific structure.

Feel free to ping me if you have any questions/thoughts. I'm still open to working on this, just right now I don't have enough bandwidth to continue.

@pksunkara
Copy link
Contributor

I am happy to see some progress on this. I would love to help, but I am swamped with a lot of other projects.

@xd009642
Copy link
Owner

@nathanwhit could you explain a bit what the build.rs is doing in the macos-v2 branch?

@nathanwhit
Copy link

nathanwhit commented Feb 24, 2020

@xd009642 Ah yeah sure, forgot about that. The following is what I've gathered from a few sources: the lldb source code for macOS, this ebook and Apple's barebones explanations.

The mach messaging/RPC interface (headers, etc.) is actually generated by a program on the system (called mig) and then linked against. In essence, some details of the interface to the mach messaging server/client is dependent on the kernel version and some other system-specific parameters (I believe, it's not well explained) and so it has to be generated on the machine which the mach messaging server/client will run. The build script just calls the mig program to generate the interface, then compiles the generated code in order to link the rust code against it.

@xd009642
Copy link
Owner

Awesome, and I've never looked at the lldb code only gdb but that seemed extra opaque with mac stuff. I'll start looking to lldb for reference when gdb is too much 😅

@xd009642
Copy link
Owner

So few issues so far, I'm working with Darwin and the Mac system headers moved but I've updated the build script. Now my current roadblock is task_for_pid always returns with a kernel error even when I run in sudo so I'm wondering if I need to sign the binary a la https://attilathedud.me/mac-os-x-el-capitan-10-11-and-task_for_pid/ so just trying to figure out how to sign the application.

@dingelish
Copy link

@xd009642
Copy link
Owner

xd009642 commented Jul 8, 2020

just found this https://github.com/kattrali/rust-mac-app-examples/tree/master/3-packaging-a-mac-app which I think shows me how to handle the permissions for an app 🎉 I'll have another play with this at the weekend

@tsauvajon
Copy link

Hi! Do you have any updates of plan for this? I've been using Docker to run Tarpaulin, but it takes much longer and forces me to have Docker up and running.

Having Tarpaulin working natively on Mac would be awesome.

@xd009642
Copy link
Owner

xd009642 commented Jun 8, 2021

@tsauvajon so this is the current approach I'm approaching for easier cross platform support #549 the only update since I last posted on that issue is I can parse and merge the profdata formats in pure rust. I just need to extract the coverage counters from the test binaries and then set about integrating it into tarpaulin. It's just a bit of a slog since for a lot of this the LLVM source code is the documentation

@xd009642
Copy link
Owner

So anyone still interested go over to llvm coverage tracking issue and try it out #549 you can install from feature branch feat/llvm-profparser-integration and add --engine llvm to your tarpaulin args and it should hopefully work. If not please mention on the llvm tracking issue and if you can share the profraw file and the log output 😁

@xd009642
Copy link
Owner

So mac has deviated from windows and I get a "permission denied" error on doc test coverage using the llvm coverage on CI. If anyone has a mac I'd appreciate some help 🙏

@TDHolmes
Copy link

TDHolmes commented Jul 1, 2022

I think it's working fine for me! Here's the output from cargo tarpaulin --engine llvm --debug run on the crate heapless:
heapless-tarpaulin-run.log

I'm on an M1 MacBook Pro. Let me know if you want me to try something specific out. I'm not super familiar with Tarpaulin yet

@xd009642
Copy link
Owner

xd009642 commented Jul 1, 2022

Great to hear it works on heapless 🎉. It should work on majority of crates given number of CI tests passing, there are some failures still on Mac CI which lives in #549 . Notable ones are doc-test coverage and following spawned programs (though that's not working on linux so I can fix that).

@Razican
Copy link

Razican commented Aug 11, 2022

In MacOS 12.5 in a MacBook with M1, running it in a private project with --engine llvm I got this panic (last commit in the branch:

thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', ~/.cargo/git/checkouts/llvm-profparser-af6f90331795d017/9d23f33/src/instrumentation_profile/types.rs:112:32
stack backtrace:
   0: rust_begin_unwind
             at /rustc/e092d0b6b43f2de967af0887873151bb1c0b18d3/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/e092d0b6b43f2de967af0887873151bb1c0b18d3/library/core/src/panicking.rs:142:14
   2: core::panicking::panic
             at /rustc/e092d0b6b43f2de967af0887873151bb1c0b18d3/library/core/src/panicking.rs:48:5
   3: llvm_profparser::coverage::coverage_mapping::CoverageMapping::new
   4: <cargo_tarpaulin::statemachine::instrumented::LlvmInstrumentedData as cargo_tarpaulin::statemachine::StateData>::wait
   5: cargo_tarpaulin::statemachine::TestState::step
   6: cargo_tarpaulin::process_handling::get_test_coverage
   7: cargo_tarpaulin::launch_tarpaulin
   8: cargo_tarpaulin::trace
   9: cargo_tarpaulin::run
  10: cargo_tarpaulin::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

@xd009642
Copy link
Owner

xd009642 commented Oct 6, 2022

So this is now merged and some point this week there shall be a new release on crates.io (probably tomorrow). closing this issue in favour of #549 I did spend all the time from around July till now fixing edge cases and panics I found on different projects so hopefully your issue is now fixed @Razican . If not head over to #549 🙏

@xd009642 xd009642 closed this as completed Oct 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BIG-TASK For tasks which seem very big. enhancement tracking-issue
Projects
None yet
Development

No branches or pull requests

9 participants