Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Remote File I/O #32

Open
daniel5151 opened this issue Oct 10, 2020 · 4 comments
Open

Support Remote File I/O #32

daniel5151 opened this issue Oct 10, 2020 · 4 comments
Labels
design-required Getting this right will require some thought help wanted Extra attention is needed new-protocol-extension New feature or request

Comments

@daniel5151
Copy link
Owner

daniel5151 commented Oct 10, 2020

See https://sourceware.org/gdb/onlinedocs/gdb/File_002dI_002fO-Overview.html#File_002dI_002fO-Overview

The File I/O remote protocol extension (short: File-I/O) allows the target to use the host’s file system and console I/O to perform various system calls. System calls on the target system are translated into a remote protocol packet to the host system, which then performs the needed actions and returns a response packet to the target system. This simulates file system operations even on targets that lack file systems.

Additionally, it would also be nice to implement Host I/O Packets at the same time (See https://sourceware.org/gdb/onlinedocs/gdb/Host-I_002fO-Packets.html#Host-I_002fO-Packets)

This has been implemented in #66. See update below...

The Host I/O packets allow GDB to perform I/O operations on the far side of a remote link. For example, Host I/O is used to upload and download files to a remote target with its own filesystem. Host I/O uses the same constant values and data structure layout as the target-initiated File-I/O protocol.


Skimming through the spec, it looks like implementing these extensions can be done in a totally backwards-compatible manner, simply by adding some new extension traits to Target, and shouldn't require any breaking API changes.

There's also some prior art from luser/rust-gdb-remote-protocol#60, which could be a good source of inspiration.


Update 8/20/2021: #66 has been merged, adding support for Host I/O packets!

In hindsight, this issue should probably have been split into two separate ones, as while there is some overlap between File I/O and Host I/O, the two are actually very different features.

@daniel5151 daniel5151 added new-protocol-extension New feature or request help wanted Extra attention is needed good first issue Good for newcomers labels Oct 10, 2020
@luser
Copy link

luser commented Jan 2, 2021

There's also some prior art from luser/rust-gdb-remote-protocol#60, which could be a good source of inspiration.

I hadn't seen this project before but it looks very cool! My crate has never gotten to a true working state, so if you have a more full-featured implementation feel free to take anything you think is useful from mine! My original goal was to see if I could write a tool that could be run against minidump files so developers could use gdb/lldb to debug minidumps from Firefox crash reports the same way that Microsoft's debuggers natively support postmortem debugging of minidumps. Given that I left Mozilla a while ago I'm not terribly invested in that anymore, although I still think it would be useful. I did cheat a bit and nerd-snipe @tromey (an actual GDB developer) into contributing, though. :)

@daniel5151
Copy link
Owner Author

Hey @luser, thanks for the heads up! Glad to know that when the time comes, gdbstub can crib your code with impunity reuse some of your excellent work 😉

Yeah, I started working on gdbstub while working on some emulators and realizing that single-stepping through raw assembly wasn't going to cut it. Over time, gdbstub's scope has expanded past that initial emulation use case, and I'm hoping to support all the major GDB RSP operating modes (I've even got a rough 1.0.0 roadmap, though that milestone is certainly still a ways away). While I don't have as much time to work on the project as I used to, I still try to find some time to hack away at gdbstub, polishing up an API here, adding a new protocol extension there, etc... Plus, it looks like some other folks both big and small have started depending on gdbstub, so it seems right to keep working on the project from time to time :)

@Sladuca
Copy link

Sladuca commented Feb 16, 2021

Hi! I recently did a very brief internship at Solana where I was trying see how to attach GDB to a eBPF VM, which Solana uses as an execution environment for smart contracts. I was able to use this crate (it's awesome btw) to do basic debugging operations without any DWARF symbols, but I ended up running into a lot of linker issues surrounding relocations that don't entirely make sense in the context of the manner in which we are using eBPF, which isn't really its intended use case (see the linked issue if you're curious).

Anyhow, if it's alright with you, I'm interested in attempting to implement Host I/O and File I/O packets as our VM has a dynamic loader that does perform relocations and it would make things much simpler if GDB could simply read executables after they have been loaded by the VM.

I should mention that I'm still rather new to open source contribution, so if I have bad etiquette or something please feel free to criticize me so that I can improve.

@daniel5151
Copy link
Owner Author

daniel5151 commented Feb 16, 2021

@Sladuca Go for it! Any and all contributions are always welcome.

If you ever need any guidance on how to structure the feature and/or how to tackle certain problems, feel free to open a draft PR with the work you've got. Alternatively, if you're more of a rapid-iteration kind of person, you can also reach me on Discord at daniel5151#4520.

And remember - gdbstub is a no_std library, so make sure that your code works without the std feature enabled!


Unrelated to the specific issue of of Remote File I/O:

  • It looks like you ended up writing your own eBPF arch definition as part of your integration. I'm always looking to include more built-in arch definitions, so if you're happy with the implementation, do consider upstreaming it!

  • ... I ended up running into a lot of linker issues surrounding relocations ...

    The first thing that sprang to mind when I read through that issue is that this could be related to Add support for responding with library relocation offsets #20. Before diving into the Remote File I/O rabbit hole, you might want to explore how the GDB Remote Serial Protocol handles relocatable binaries, and see if implementing those bits of the protocol might fix your specific problem.

@bet4it bet4it mentioned this issue Jul 30, 2021
13 tasks
@daniel5151 daniel5151 added the design-required Getting this right will require some thought label Sep 25, 2021
@daniel5151 daniel5151 changed the title Support Remote File I/O (+ Host I/O) Support Remote File I/O Nov 16, 2021
@daniel5151 daniel5151 removed the good first issue Good for newcomers label Nov 28, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
design-required Getting this right will require some thought help wanted Extra attention is needed new-protocol-extension New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants