-
Notifications
You must be signed in to change notification settings - Fork 5
/
RELEASE_NOTES
446 lines (403 loc) · 22.5 KB
/
RELEASE_NOTES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
$Id$
3.42 2024/06/04 (cvuosalo)
- Add support for https in URLs specified by the FileBaseDirectory
configuration variable.
3.41 2019/12/31 (dwd)
- When a "X-Frontier-Opts: DontCacheErrors" request header is set,
abort error responses before they send the zero-length http/1.1
last chunk. This prevents proxies from caching the response.
3.40 2018/08/20 (dwd)
- Ignore any text before the BEGIN line in the private key file.
This appears to have been done in 3.33 but lost in 3.34.
3.39 2018/01/23 (dwd)
- When responding with an error condition before headers are sent,
instead of setting a Last-Modified header to be blank, set it
to the oldest possible time of midnight on January 1, 1970.
With the old behavior, Squid-3 does not clear the cache even
though Squid-2 does. With the new behavior, squid-3 always
clears the cache even when another error sends the same
Last-Modified time.
3.38 2017/05/24 (dwd)
- Add log message after DB data transferred including the msecs
it took to transfer the data.
3.37 2017/03/22 (dwd)
- Fix bug introduced in 3.36 that caused a null pointer exception in
in the FilePlugin.
3.36 2017/02/08 (dwd)
- Support HTTP 1.1 chunked encoding to backend FilePlugin http servers.
Note that the protocol to the client can now send multiple byte
arrays per request instead of just one. An implication is that
fn-fileget from frontier-client versions older than 2.8.21 will not
always retrieve all of the data when the backend is an http server
(although it will when reading directly from files).
- Support MySQL as a database backend in the SQLPlugin. This required
changing the Last-Modified handling to use the MySQL table that
always keeps track of modification times on all tables (it does not
require an add-on function like oracle does).
3.35 2016/10/09 (dwd)
- Add new servlet option ExpireEmptyQueriesLikeErrors, default false,
which when true causes empty queries to be expired at the same shorter
age as errors (default 5 minutes). Previously this was always true.
- When ExpireEmptyQueriesLikeErrors is true, mark the returned payload
with the shorter max_age so if a cached item age exceeds the time,
the client will repeat the request with the shorter max age. This
can happen because a later refresh with If-Modified-Since when the
Last-Modified time hasn't changed, the regular long expiration time
will be returned because the query is not repeated.
3.34 2014/10/29 (dwd)
- Support host private keys (for digital signatures) that start with
'BEGIN PRIVATE KEY' in addition to 'BEGIN RSA PRIVATE KEY'. That
form has begun to be used at CERN with 4096-bit RSA keys; the
old form was previously used for 2048-bit keys.
- Eliminate an infinite loop when there's an '&' in an error message
as sometimes happens with the FilePlugin.
- Do one less level of url decoding on the query string put into
catalina.out logs, because one level is already done and that
better matches the URL seen by a client using the FilePlugin.
3.33 2013/12/26 (dwd)
- Added new "forever" time to live, a third cache expiration time
requested by clients, in addition to "short" and "long" times to
live. Added config.properties option ForeverCacheExpireSeconds
which defaults to a year of seconds. If a "forever" query returns
empty results, it is considered to be an error.
- Reduced the time that empty query results will be cached for
"short" or "long" time to live queries to a maximum of 5 minutes
or the "short" time, whichever is smaller.
- Added a "max_age" value at the end of the response if the http
header has already been sent with a longer Cache-control: max-age
than is later determined to be necessary. The things that can
cause the http header to be sent are either a keepalive message
because the request had to wait longer than 5 seconds to read
from the database, or at least a buffer full of data was already
sent before an error occurred. The events that can reduce the
max age late are some kind of error with the database (which
used to be handled by only signalling "global" errors to the
client) or an empty result from a database query.
- Show a more user friendly error message if the private key file
cannot be decoded.
- Added removal of any excess text from before the "BEGIN" line in
the host certificate.
3.32 2013/10/22 (dwd)
- Add support to the FilePlugin for reading from http URLs in
addition to files on the local disk. If FileBaseDirectory
begins with http://, this new feature is enabled. If the http
query returns a Content-Length header, the response is streamed
through to the client, otherwise the response is read into a
memory buffer to determine the length. Last-Modified headers
are passed through from the backend http server to the client,
and If-Modified-Since is passed through from the client to the
backend http server.
- Fix bug in the FilePlugin that prevented it from sending the
Last-Modified header for files.
- Add queuing and keepalive support to the FilePlugin, similar to
the SQLPlugin. The maximum number of simultaneous connections are
specified in the new config.properties option MaxFileConnections,
default 5. The options MaxDbAcquireSeconds and MaxDbExecuteSeconds
then become relevant for sending keepalive messages while waiting,
even though it isn't technically a "DB". When reading files (as
opposed to http urls) the time to "execute" is instantaneous
because it just opens the file, so MaxDbExecuteSeconds isn't
important then, but when reading http urls the execute time is
significant because the http server can take a while to respond.
- Improve the accuracy of the active connection count in the log by
moving the counter decrement to be before the connection release.
- Remove old unused code that supported reading an "xsd_type" from
a database table whose name was specified in config.properties
option XsdTableName, and that executed a plugin based on the name
read from the database. Also remove related config.properties
option UseFdoCache.
- Remove old unused code related to MonAlisa monitoring, including
config.properties options MonitorNode and MonitorMillisDelay.
3.31 2013/10/21 (dwd)
- Fix off-by-one error in FilePlugin that had the effect of
allowing ".." in the file path.
3.30 2013/06/28 (dwd)
- Add support for sending digital signatures on responses instead
of an md5 hash, when the client includes the URL parameter
"&sec=sig". Also add support for sending the host certificate
when the client requests it as type "cert_request". Requires
the certificate to be specified in a new parameter 'CertFileName'
and the key in a new parameter 'KeyFileName'.
- Change to just print one error message to the log when the
client drops the connection instead of multiple stack traces.
- Change to not print a stack trace when there's an error getting
the last-modified time.
- Add msecs= on the DB connection acquired message to show how
long it took to acquire the connection.
- Change internal errors to force a restart of the servlet just
like database close problems do.
3.29 2011/05/17 (dwd)
- Fixed a bug that corrupts the response when a query for a
last-modified time timestamp takes more than 5 seconds, causing
a keepalive to be sent. The bug results in the client seeing an
error message:
XML parse error 7:mismatched tag at line 7
Now if the query takes too long the result is discarded, because
it is too late to insert the Last-Modified header after a
keepalive is sent.
- Add the number of active and remaining connections to the DB
connection acquired and released log messages, respectively.
3.28 2011/03/24 (dwd)
- Add support for loading locally-accessible files in addition
to database queries. Add new FileBaseDirectory property to
indicate the base directory of the files that the servlet will
load. Make the DataSourceName property no longer required; now
at least one of DataSourceName and FileBaseDirectory are
required. Corresponding support in the client will be in
frontier_client 2.8.1.
3.27 2011/01/20 (dwd)
- Change the expiration time on the 503 SERVICE_UNAVAILABLE error
to be the time used for errors (the lesser of 5 minutes and the
short cache expiration time) instead of the long cache expiration
time.
3.26 2011/01/12 (dwd)
- Add new SendStaleIfError property, default true, which will send
the Cache-control header "stale-if-error" if the client sends in
Cache-control: max-stale. This prevents squid from re-using
expired data when there's a server error (by default it will
reuse data that's up to a week old). Setting max-stale on the
client side doesn't affect squid but it indicates that the
client is ready to deal with the server errors (frontier_client
has been doing that since October 2008 but I never enabled the
server side because I thought there was a problem -- details in
squid bug #2481).
3.25 2010/12/09 (dwd)
- Add new MaxThreads property to be the maximum number of threads
that the servlet will allow to wait for a database connection.
If a NOT_MODIFIED response can be sent quickly without
contacting the database, that will still happen but otherwise if
the database needs to be contacted and there are too many
threads running the response will be SERVICE_UNAVAILABLE.
Default 100.
- Ignore exceptions when writing out the transaction end xml.
This was causing a MUST NOT HAPPEN HERE exception in cases when
the connection had already been dropped by the client and
already been reported as an exception.
- No longer require the XsdTableName property to be set, as it
isn't used by current deployments.
3.24 2010/07/28 (dwd)
- Clearly identify the license for the package to be the open
source (BSD) Fermitools license.
- Support having a slash after the 'Frontier' in the URL in
place of the question mark. A slash is better because
by default squid does not cache URLs with question marks
in them so with slashes a standard squid configuration
can be used instead of a custom one.
- Add elapsed time to the DB execute log message
3.23 2010/02/12 (dwd)
- Fixed race condition in the assignment of thread id numbers.
This has been happening for a long time but is now more
important because the thread names are used for cleaning up
keepalive timer threads. Also include the servlet name in the
thread names to ensure no clashes of names between servlets
(which has not been seen in practice, it is just in case).
3.22 2009/11/25 (dwd)
- There was a serious flaw with the implementation of sending
keepalives during DB execute: it was only able to track one at
a time, and as a result multiple simultaneous connections to the
DB caused a null pointer exception and soon was unable to
connect to the DB anymore. It's now able to track any number of
simultaneous connections.
- Added -Xlint:unchecked javac compile option, and cleaned up all
the warnings it added to unchecked generic types.
- Made the indentation on keepalive XML elements prettier.
3.21 2009/11/02 (dwd)
- Changed the parsing for multiple table names. Now looks for an
optional table alias followed by a comma to indicate that another
table name is coming, rather than searching for a specific list of
keywords. The keywords were matching portions of some table names
in CMS.
- Added support for continuing to send keepalive messages to the
client while waiting for the database to execute a query. This
was needed in ATLAS because it has some queries that can take
over 10 seconds to execute the first time, until it gets into an
Oracle cache. Added config.properties option MaxDbExecuteSeconds
to control the amount of time allowed, default 10 seconds. The
frontier_client's own readtimeoutsecs option (which also defaults
to 10 seconds) is added to that before it times out.
- Added an additional config.properties option MaxDbAcquireSeconds
which is how long to send keepalives while waiting to acquire the
database connection. Defaults to 300 seconds which was the value
previously hardcoded.
- If a query contains the form "WHERE OWNER=''" (which happened once
in CMS) then no longer attempt to look up a modification timestamp
for that owner, just let the query proceed without a timestamp.
3.20 2009/10/02 (dwd)
- Added a global error to a response where one had inadvertently
been left out, after a failure acquiring the database. Global
errors cause the client to force a refresh to clean the error
out of the cache, and a case was found after a database overload
last week where a 'Timed out waiting to acquire the DB' error
was cached indefinitely because it wasn't marked as a global
error.
3.19 2009/07/14 (dwd)
Changes for this release primarily written by David Front.
- Support queries on multiple tables at once when tracking
modification times. This is needed in order for ATLAS to use
Frontier. When multiple tables are in a query, the timestamps
on all the tables are looked up in a single database query and
the most recent timestamp is cached.
- Add new config.properties option 'VerbosityLevel'. When set to
2 or higher, add source file name and line number to each log
file line. When set to 4 or higher, add many debug messages.
3.18 2009/05/21 (dwd)
- If there's an exception when releaseing the database connection, force
the servlet to restart by touching WEB-INF/classes/config.properties.
That's quite ugly, but I couldn't think of a better way to do it. We
saw a case where this happened and it ended up hanging the servlet and
eventually the whole tomcat, so it is important to take of this.
NOTE: if installed by root but run under a different user id, as
is often the case when it is installed by rpm, this approach will
not work!
3.17 2009/02/26 (dwd)
- Add new configuration variable LastModifiedTableName that
defines the name of the single table where modification times
are kept for the whole database, rather than reading them
from a LAST_MODIFIED_TIMES table in each account. This is
now required if ValidateLastModifiedSeconds is set.
3.16 2008/12/19 (dwd)
- Log 'modified at time' also when the modification time was
cached
- Log 'using cached last-modified time of' the account.table
when a cached modification time was found
3.15 2008/12/17 (dwd)
- Distinguish in the log when an item has been modified after an
if-modified-since by noting it as 'modified at time'.
- When the X-Frontier-Id header is missing, still put in the
initial log entry as much as is known so we can find out where
the request is coming from.
3.14 2008/12/04 (dwd)
- Avoid accumulating threads indefinitely when a database is down,
by maintaining a lock for getting connections to the database and
raising an exception immediately if the 5-minute keepalive
thread had already given up by the time the lock is acquired.
Previously all the threads would in turn wait 6+ minutes to try
to open a socket to the database server and would tie up so many
resources that they began to interfere with connections to other
databases.
- Avoid a couple null pointer exceptions that happened when
required pieces were missing from a request. Instead, raise
more helpful exceptions for "no query string" and "no
X-Frontier-ID".
- Also avoid a null pointer exception leading to a "MUST NEVER
HAPPEN HERE" internal error when there's no stack trace
available in an exception.
- Remove an extraneous blank in exception log entries that
sometimes made the log appear as if it had an extra blank line.
3.13 2008/11/05 (dwd)
- Make sure that a '?' bind variable substitution exists in query
before looking for colons, because colon is occasionally used in
queries as part of a table value.
3.12 2008/11/04 (dwd)
- Add new configuration variable ValidateLastModifiedSeconds which
defines the number of seconds between querying for modification
times in a LAST_MODIFIED_TIMES table in the same account as
each query. The modified time of each table is cached
internally for that number of seconds. The time is used to send
a Last-Modified header. Squid in turn uses that value once
the item expires to send an If-Modified-Since header. When
that is received, if the time has not changed send back
an http '304 NOT MODIFIED' response. This all combines to
allow much more frequent object expirations without overloading
the frontier infrastructure, because most times very little
data is transferred, only the query and NOT MODIFIED. This
requires that every account loaded with this option have
a LAST_MODIFIED_TIMES table kept up to date. One way is
provided to do this in Oracle, using a PL/SQL script called
setup_modtimes.sql.
- Use "Cache-Control: max-age" instead of "Expires" headers in
the response. Squid handles both and the latter are more
difficult to calculate and manage.
- Support bind variables in queries, with a question mark where
each variable is to be inserted. The values for the variables
must be appended to the query, separated by colons (:). This
helps performance a bit at least on Oracle, because it is able
to cache the parsed value of the query and re-use it rather than
caching many similar queries.
- Handle global errors more robustly, so they should always be
able to be parsed by the client. Previously some kinds of
errors would result in invalid xml code in the response.
- Give more debug information in the log if the command parser
sees a premature end of command.
3.11 2008/04/14 (dwd)
- Set the maximum keepalive time (which includes the time it takes
to allocate a connection to the DB) to 5 minutes
3.10 2008/02/28 (dwd)
- Set the expiration time for exception messages to 5 minutes.
3.9 2007/10/24 (dwd)
- When a blob is null, send a null indication rather than empty.
This is the more correct way to handle null blobs than what was
put in in release 3.5.
3.8 2007/05/24 (dwd)
- Back out change from 3.7 because that ends up eliminating the
Content-Length header from even small responses, causing squid
to drop the connection after every query instead of keeping
persistent connections. Instead, we will rely on squid 2.6's
"collapsed_forwarding" feature to maximize sharing. Also fix an
inadvertent too-early sending of the response header that was
put in in version 3.5. Add a log message about the response
being precommitted whenever the header is committed early,
resulting in the removal of Content-Length. This now happens
only when responses take up more than the default 8K buffer in
the ServletOutputStream.
3.7 2007/05/01 (dwd)
- Send response headers as soon as possible to maximize the
sharing that squid does.
3.6 2007/04/05 (dwd)
- Get the transaction id (with a -ka suffix) into log messages
from the keepalive thread
3.5 2007/03/22 (dwd)
- Cleaned up log messages so those that occur regularly all will
show timestamp and thread id
- Greatly improved performance for blobs, for small rows, and for
zipping
- Replaced Base64 encode/decode implementation with much simpler/
cleaner version
- Eliminate null pointer exception on null blobs
- Renamed CacheExpireSeconds and CacheExpireHourOfDay to
LongCacheExpireSeconds and LongExpireHourOfDay. Added two new
parameters ShortCacheExpireSeconds and ShortCacheExpireHourOfDay
which are the same except apply to all queries that have
"&ttl=short" (ttl stands for time-to-live). The default on
ShortCacheExpireSeconds is 0.
- Catch all Throwables, not just Exceptions, so that Errors are
also included and debugged
- When an Exception or Error occurs pass the function, source file
number, and line number to the application in addition to the
name of the throwable
- The "stop" log message now includes the entire transaction, so
the number of active threads and the elapsed time are more
meaningful
3.4 2006/10/26 (dwd)
- Added support for oracle BLOB (Binary Large OBject) types
- Added two optional parameters to WEB-INF/classes/config.properties:
CacheExpireSeconds - number of seconds after queries in which
the caching of the query should expire. Default is 7 days of
seconds, 604800.
CacheExpireHourOfDay - the hour of the day at which all
query caching should expire, if less than CacheExpireSeconds.
For example, if it is set to "23", all items will expire at
11:00 p.m. in the local time of the server. Default is
to use CacheExpireSeconds instead.
3.3 2006/09/05 (dwd)
- Fixed bug where the keepalive message task wasn't shut down if
the database connection failed
- Added a refinement to Luis Ramos' patch of 3.1: don't add the
parenthesized precision after the NUMBER data type if the
precision is zero
3.2 2006/07/26 (dwd)
- Added sending a keepalive message to the client every 5 seconds
while waiting to acquire a database connection, to avoid client
timeouts under heavy load
- Added sending the servlet version to the catalina log and the
client
- Reduced non-useful debugging output to the catalina log and
improved some others
3.1 2006/06/07 (dwd)
- Added feature to zip the retrieved payload using gzip levels 0-9
where level 0 is no zipping, level 1 is fastest but least
compressed, and level 9 is slowest but most compressed
- Added precision in parentheses to the NUMBER data type. Requested
by Luis Ramos of CERN and patch supplied by him
- Cleaned up some exception message typos
3.0 and earlier releases did not have release notes