AttributeError from twcgi dependent on timing of connection lost and process ended

[twisted-trac]

	@chrisbarber reported
Trac ID	trac#9468
Type	defect
Created	2018-06-13 17:42:17Z

Running a twisted cgi resource (inside crossbar.io, in this case) with a simple script, and GETting it from a browser, cURL, etc. caused no problems. Requesting the page from an f5 load balancer "health check" caused the following traceback (not causing any consequences in this case other than polluting the crossbar log):

2018-06-11T15:27:26-0700 [Router      31860] unexpected error in processEnded
Traceback (most recent call last):
  File "/local1/miniconda3/envs/datacube/lib/python3.6/site-packages/twisted/internet/process.py", line 64, in reapAllProcesses
    process.reapProcess()
  File "/local1/miniconda3/envs/datacube/lib/python3.6/site-packages/twisted/internet/process.py", line 350, in reapProcess
    self.processEnded(status)
  File "/local1/miniconda3/envs/datacube/lib/python3.6/site-packages/twisted/internet/_baseprocess.py", line 52, in processEnded
    self.maybeCallProcessEnded()
  File "/local1/miniconda3/envs/datacube/lib/python3.6/site-packages/twisted/internet/process.py", line 987, in maybeCallProcessEnded
    _BaseProcess.maybeCallProcessEnded(self)
--- <exception caught here> ---
  File "/local1/miniconda3/envs/datacube/lib/python3.6/site-packages/twisted/internet/_baseprocess.py", line 64, in maybeCallProcessEnded
    proto.processEnded(Failure(reason))
  File "/local1/miniconda3/envs/datacube/lib/python3.6/site-packages/twisted/web/twcgi.py", line 321, in processEnded
    self.request.unregisterProducer()
  File "/local1/miniconda3/envs/datacube/lib/python3.6/site-packages/twisted/web/http.py", line 913, in unregisterProducer
    self.channel.unregisterProducer()
builtins.AttributeError: 'NoneType' object has no attribute 'unregisterProducer'

This can be reproduced by requesting the page like this from python:

from __future__ import print_function
import socket
clientsocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
clientsocket.connect(('localhost', 8080))
clientsocket.send('GET /cgi/apptest.py HTTP/1.1\r\n\r\n')
ret = clientsocket.recv(10000)
print("Returned\n" + ret)

Sleeping for some time at the end of the script fixes the problem so it seems to be down to timing, and from what I can tell Request.connectionLost (which sets the request's channel to None) is probably getting called before CGIProcessProtocol.processEnded (which calls Request.unregisterProducer that will fail when channel is None).

Searchable metadata

trac-id__9468 9468
type__defect defect
reporter__chrisbarber chrisbarber
priority__low low
milestone__None None
branch__ 
branch_author__ 
status__closed closed
resolution__fixed fixed
component__web web
keywords__None None
time__1528911737435288 1528911737435288
changetime__1596777721458454 1596777721458454
version__None None
owner__glyph glyph

[twisted-trac]

	@twm commented

chrisbarber also provided a PR: #1027

[twisted-trac]

	@glyph set owner to @glyph

[twisted-trac]

	@glyph set status to closed

In changeset 78032af

#!CommitTicketReference repository="" revision="78032afbc70639a2eafdc57933140eb76465f85d"
Merge pull request #1027 from chrisbarber/trunk

Author: chrisbarber

Reviewer: glyph

Fixes: ticket:9468

twisted.web.twcgi.CGIProcessProtocol.processEnded(...) now handles an already-finished request, for example when request.connectionLost(...) was called previously.