No Control of Mars Pro Printer at Firmware 4.4.3

[BlueFinBima]

I'm a new user (ie Mariner3D has never worked completely for me) I get the #180 symptom, however I do not get any control of the printer which seems different from the experiences of others who get this symptom.

Looking at the serial data being sent from the Pi Zero W to the Mars Pro, I see the following
M23 /first file.ctbM4006M4006M4006M4006M23 /second file.ctbM4006M4006M4006M4006M4006M4006M4006M4006
that is, a constant stream of commands without CR LF.

In order to rule out my serial monitor, I performed the following on the Pi Zero W
cat 1.txt > /dev/ttyS0
where 1.txt contains three words separated by CR LF.

I see the following in the serial monitor

My conclusion from this is that Mariner3D / Python is not sending the CR LF even though I can see from the Mariner code that it should be sent.

Assuming my observations are reliable, then I think that the commands from Mariner cannot be digested by the printer. I cannot escape the fact that nobody else sees this problem so it is likely to be a problem of my own making rather than Mariner.

A short description of what the bug is that the printer cannot be controlled via the UART.

Expected Behavior
I expect that the printer will be sent commands terminated with an "\r\n". I can see in printer.py that this is the intent of what Mariner3D is doing. I don't see why this seems not to be happening.

There is nothing of interest in the Mariner3D log.

Details:

• Mariner Version: 0.2.0-1
• Printer Model: Elegoo Mars Pro V4.4.3_c1_LCDF System EL3D-3.0.1
• Raspberry Pi Model: RPi Zero W with raspios_lite_armhf-2021-05-28/2021-05-07-raspios-buster-armhf-lite

Other
cmdline.txt is
root=PARTUUID=82934f69-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait modules-load=dwc2
[all] clause of config.txt contains

dtoverlay=dwc2
enable_uart=1

Python is 3.7.

[BlueFinBima]

I installed Mariner on a Pi3B and can also reproduce the same problem there. Additionally, I run the following script, designed to mimic what Mariner's printer.py does, and with both this script and mariner communicating on the serial port, I see the \r\n in the serial monitor for my python script, but not from mariner.

'''
UART communication on Raspberry Pi to mimic Mariner comms
'''
import serial
from time import sleep

ser = serial.Serial ("/dev/serial0", 115200)                      #Open port with baud rate
while True:
    sleep(1)
    ser.write((f"M23 /W1bble.ctb").encode() + b"\r\n")                #transmit data serially 115200

I am completely at a loss to understand what I am seeing unless the mariner I am running is not the same as the code on github.

Hopefully this additional information will help someone more knowledgeable than I to point me in the right direction.

Many thanks

[BlueFinBima]

Answering my own question, I've just seen commit 599f4b0 so I guess I am running a version of mariner from before this. Is there a Debian package which includes the latest mariner code?

[luizribeiro]

Hey, I'm still not sure why mariner doesn't seem to be sending line breaks for you. I'll get back to this thread once I have a bit more of time, but thanks for your investigation so far!

Is there a Debian package which includes the latest mariner code?

Yes, every commit on GitHub has an Action that runs on CI for packaging that build as a .deb file, so you can download it from the artifacts of that GitHub Action run. The current one is here: https://github.com/luizribeiro/mariner/actions/runs/1018824027

That said, I don't expect the latest commit on master to be of any help to this issue from what we have seen so far

[BlueFinBima]

Development build of the latest github code now shows the CR LF. Miniterm shows that the printer can be controlled via the UART, but M4006's always seem to be rejected with the increasing 'N:xx' response. The Mars Pro also gives an error to M4002 (get firmware version).

[BlueFinBima]

Yes, every commit on GitHub has an Action that runs on CI for packaging that build as a .deb file, so you can download it from the artifacts of that GitHub Action run. The current one is here: https://github.com/luizribeiro/mariner/actions/runs/1018824027

Thanks!

[BlueFinBima]

Not a big surprise, but the new .deb gives the same results as the development build ie the commands correctly have a CR LF at then end of them. Unfortunately, this does not solve the problem of not being able to print from mariner.

My test scenario is that I select a file in mariner, and press "Print". Nothing happens on the printer.
I then stop the mariner service and fire up minicom and a shell.
In the shell, I issue
echo "M23 /File1.ctb" > /dev/serial0
and in minicom I see

File selected
File1.ctb Size:8472210
File selected
ok N:nnn

nnn is an incrementing number.

I also see plain old "OK"'s appearing regularly which I presume is some sort of heart-beating.

M20, M24, M25, M27 and M33 work as I would expect.

From what I see, OK's with an "N:" are a response to a command (as opposed to plain old "OK") so they should definitely not generate an exception.

One point I will make is that I recently updated my Mars Pro to V4.4.3_c1_LCDF System EL3D-3.0.1 which was a requirement for ChituBox 1.9. Not sure if this is relevant, and I would rather not remove this firmware, but I thought I would highlight this.

Next stop is to look through printer.py to see if I can determine why my Mars Pro dislikes what mariner is sending.

[BlueFinBima]

So....
I have to presume that the code in printer.py is working for a good many printers/firmware levels, however for my Mars Pro at 4.4.3, the way that printer.py works today is inconsistent with the results I see from my manual testing (IMHO).

If I want to print a file, I need to do the following:

M23 /File1.ctb ->
                        <- File selected
                        <- File1.ctb Size:8472210
                        <- File selected
                        <- ok N:460
M24                ->                                # Resume
                       <- ok N:461
### Printing now starts
M25                ->                                # Pause
                       <- ok N:462
M27                ->                                # Print Progress
                      <- SD N:463
                      <- byte 1363986/8472210
                      <- ok N:463
M33               ->                              # stop print
                      <- ok N:464
### Printing stops

For all of the M400x codes that I have issued, I only get "ok N:nnn" responses and nothing else.

I also think that if commands arrive at my printer too quickly, then the printer can get into a strange state and appear to become unresponsive and it can take a bit of coaxing to get the printer to action commands again.

@luizribeiro My conclusion is that for Mariner to work with my Mars Pro, printer.py would need a reasonable amount of rework. For printer.py to work with my Mars Pro and continue to work as it does today for all of the printers currently supported, then I think there is an even bigger task ahead. I'm interested in your thoughts.

I also don't think printer.py can afford to ignore the line numbers on the responses coming back from the printer which are resulting in exceptions at the moment.

NB M6030 is needed to start the print and change the screen to the "printing screen" so that is obviously better than the M24 in the example above.

[BlueFinBima]

Notes: Mainly using https://docs.google.com/document/d/14UBMO0Vhh9Lr0V3xcdetQ2_4UDnjFnho7OnbNxLOs3o/view# gcode reference.

[BlueFinBima]

I've forked Mariner, and my forked Mariner is mostly working for Mars Pro firmware 4.4.3. There are a few issues to be ironed out, but it is mostly working with a few kludges necessitated by the lack of information from M400n commands. The code is intolerant of earlier firmware so not fit for a PR. Several of the tests are also broken. Finally, I've removed the need to have the #180 fix.

My conclusions from further testing is that the "new" UART code in my printer's firmware is a little buggy when it comes to output buffer management for multi-line responses. As as example, the text response to M27 should be something like
printing byte 0/29771388
but it looks to me that they have simply put the preceding
SD N:10\r\n
into the same buffer at offset 0, with the result that what is printed is

SD N:2
ng byte 364998/8472210
ok N:2

or

SD N:10
g byte 364998/8472210
ok N:10

or

SD N:100
 byte 364998/8472210
ok N:100

I also get pretty messed up file lists from M20

If the UART is important to ChiTu, I can only imagine that they will fix these bugs with the result that any mariner code written to accommodate these bugs in the responses is likely to break in the future.

My thinking is that the newer firmware levels should be monitored for a while to see if further changes are made (or rolled back) so unless there is increasing pressure from users who install the problematic firmware levels, I'd suggest not making lots of changes as a result of my experiences. Happy to discuss further.

My code is mostly in printer.py https://github.com/BlueFinBima/mariner/blob/master/mariner/printer.py

[BenStein1]

@BlueFinBima I'm on the same new firmware on my Mars 2 Pro and just installed your fork. Seems to be working well so far. Ran my first print entirely from the web interface for the first time, looks like it's holding up so far. Thank you @luizribeiro for creating this project And BluefinBima for the update. I'll let you know if there are any snags in the new code for me.

[BlueFinBima]

@BenStein1 thanks for the feedback. I have opened a few issues on my fork describing the problems I know about. I do not remotely suggest that the current inventory is the extent of the problems.
https://github.com/BlueFinBima/mariner/issues
I am happy for you to raise additional issues, but my intention is that my fork is a temporary measure to help people (and myself) until the official Mariner is able to support the later Firmware levels, so no guarantees that I will fix new issues.

[luizribeiro]

Thanks for digging into this @BlueFinBima! I was looking at the changes on your repo and I think it should be possible for us to just create a separate ChiTuPrinter class which handles these more modern firmwares. Then we can expose an option on the config file to toggle between the two implementations.

It will lead to some duplicated code and I guess the fact that firmware 4.4.3 doesn't support certain commands is a bit unfortunate, but we do what we can :)

If you or anyone has interest on merging these changes in I'm happy to review a PR

[BlueFinBima]

@luizribeiro Thanks for taking a look at this. I still feel that further changes by Chitu in this area are likely / possible so with my current workload, I personally won't revisit this until yet another firmware is released which is incompatible with this Mariner3D.
I'm not averse to creating a PR in future, however the technologies used in this project are all alien to me, so I am unlikely to be able to include automated tests.
I will continue to raise issues against my fork to document problems in case someone else wants to create a PR for 4.4.3.

[J-Iszatt]

Hello @BlueFinBima and @luizribeiro,

As I was also facing troubles with the specific firmware mentioned above (running on a mars pro 2), I finally managed to find this great explanation (and the even better solution) here!
I am extra thankful for the original mariner software as well as the additions to the printer.py file and was wondering if a link or at least a hint to problems with 4.4.3 could be added to the readme of this project? Given that more and more users are upgrading, I am relatively sure, that this would save people a lot of time :P

On top I want to ask for a couple of lines on how to get the code running by ‘git-cloning’ it (at least for Bima’s repo, there are no .deb files available).

Thanks again for all the great work! Please keep it up.
Cheers
JI

[fcollingwood]

I thought this might solve the issue of encrypted .ctb files causing 500 errors. Unfortunately, not.

[BlueFinBima]

Hi, I have not used my SLA printer for a while so I have not encountered the problem you describe, however, from the outset it appeared that the internal firmware was in a state of flux, and I guess that this continues with the encryption. Fundamentally, the encryption should not in itself be a problem because the printer firmware knows how to decrypt it, but if the information about the print being exposed on the serial port is different or worse, non-existent, then this could spell the end of mariner. Unfortunately, until someone more knowledgeable than I fixes the build problems, changes on my fork are not going to happen.

…

On Sun, 16 Jan 2022 at 11:32, fcollingwood ***@***.***> wrote: I thought this might solve the issue of encrypted .ctb files causing 500 errors. Unfortunately, not. — Reply to this email directly, view it on GitHub <#453 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AENLAGGD3QTRPZ7OEFFCQPTUWKUEXANCNFSM5CGK7BTQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[Blanks82]

Can anyone explain how to install this forked version over the original? The docs listed point to the original mariner & I dont know enough about this to make it install this fork instead.

I presume its fairly simple, thanks for any help.

[koshisan]

@Blanks82 I ran into the same problem with my Mars 3. It seems there isn't much development happening on this anymore...

I tried to replace the .py files in /opt/venvs/mariner3d/lib/python3.9/site-packages and /opt/venvs/mariner3d/lib/python3.7/site-packages (don't even know at this point what directory should be used lol) with the ones from @BlueFinBima's fork. However, the behaviour stays the same for me:

Webfrontend throughs a HTTP 500 when anything file-related is happening. Unfortunately reporting the status via the (undocumented) API seems to count as file related...

Even through the serial line is still being read and there is info, because of the file read function not working, the client doesn't get the reply:

DEBUG:mariner.printer:Obtain Print Status Command Requested
DEBUG:mariner.printer:b'M4000\r\n'
DEBUG:mariner.printer:Full Response Message:
ok B:0/0 X:0.000 Y:0.000 Z:49.950 F:256/256 D:15504575/18035958/0
DEBUG:mariner.printer:Obtain the Selected File Command Requested
DEBUG:mariner.printer:b'M4006\r\n'
DEBUG:mariner.printer:Full Response Message:
ok 'Butt-notail-PLUG-PANTY_Rl.ctb'

ERROR:waitress:Exception while serving /api/print_status
Traceback (most recent call last):
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/waitress/channel.py", line 397, in service
task.service()
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/waitress/task.py", line 168, in service
self.execute()
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/waitress/task.py", line 434, in execute
app_iter = self.channel.server.application(environ, start_response)
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/flask/app.py", line 2464, in call
return self.wsgi_app(environ, start_response)
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/whitenoise/base.py", line 85, in call
return self.application(environ, start_response)
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/flask/app.py", line 2450, in wsgi_app
response = self.handle_exception(e)
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/flask/app.py", line 1867, in handle_exception
reraise(exc_type, exc_value, tb)
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/flask/app.py", line 2447, in wsgi_app
response = self.full_dispatch_request()
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/flask/app.py", line 1952, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/flask/app.py", line 1821, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/flask/_compat.py", line 39, in reraise
raise value
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/flask/app.py", line 1950, in full_dispatch_request
rv = self.dispatch_request()
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/flask/app.py", line 1936, in dispatch_request
return self.view_functionsrule.endpoint
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/mariner/server/api.py", line 73, in print_status
sliced_model_file = read_cached_sliced_model_file(
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/flask_caching/init.py", line 952, in decorated_function
rv = f(*args, **kwargs)
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/mariner/server/utils.py", line 23, in read_cached_sliced_model_file
return file_format.read(config.get_files_directory() / filename)
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/mariner/file_formats/ctb.py", line 154, in read
ctb_slicer = CTBSlicer.unpack(file.read(CTBSlicer.get_size()))
File "/opt/venvs/mariner3d/lib/python3.9/site-packages/typedstruct.py", line 89, in unpack
return cls(*unpack(cls.get_format(), buffer))
struct.error: unpack requires a buffer of 76 bytes

Maybe I will just delete the file handling stuff, so I can at least get a status reply...

[BlueFinBima]

@Blanks82 the deb got deleted from the packaging action, and I was unable to get a successful package to recreate it. I have now finally managed to get this to work (I think) and the deb is now available here https://github.com/BlueFinBima/mariner/releases/tag/v0.2.0-1

[jremen]

@BlueFinBima

Have you been able to test your fork with newer firmware versions such is 4.5.1?
I would like to test this with my Saturn S. Plenty of experience with Unixes and shell, some programming experience as front end dev (however, mainly JS and React). Using Klipper on FDM printers now…
I have OTG and Samba parts working correctly, main fork of Mariner is working (had not connected it to printer yet).
Later I want to add Raspberry Pi cam module and modify React frontend to get image from RPI camera output to monitor prints.
Being able to control printer too would be absolutely perfect.

[BlueFinBima]

@jremen I have not used Mariner or my SLA printer for quite some time (there was a major resin spillage which dampened my enthusiasm for the technology). Anyway, my fork is dormant, at least until I have a need to use the machine again, and from the experiences of others, I probably won't update the firmware unless there is some very compelling reason to do so. Good luck with your project.

[mnbf9rca]

@BlueFinBima I merged your work with @Desterly's downstream changes to read encrypted CTB files. I had to update the build workflow too. It's now here. Happy to raise a PR back on your fork or keep it here?

[DimitriGaming-png]

@BenStein1 thanks for the feedback. I have opened a few issues on my fork describing the problems I know about. I do not remotely suggest that the current inventory is the extent of the problems. https://github.com/BlueFinBima/mariner/issues I am happy for you to raise additional issues, but my intention is that my fork is a temporary measure to help people (and myself) until the official Mariner is able to support the later Firmware levels, so no guarantees that I will fix new issues.

I had an issue that wouldn't allow me to upload files on the web and got your fork, now I can upload files and preview them but I don't have the ability to start a print on the web or view printer status while its printing (i can manually start the print that I uploaded on the web and it prints just fine).
Additionally, when I attempt to start a print from the web I get the error "The printer returned an unexpected response: 'ok N:0\r\n'". It appears that although your fork solved my original solution it has given me the issue that this thread started with. I am thankful I can now upload files but it would be awesome if I could start the print and view status too. Should be noted that I have no idea what I'm doing when it comes to this stuff and it is possible that I didn't install your fork correctly (all I did was follow the install instructions in the zip). Any help would be appreciated, thanks!