-
Notifications
You must be signed in to change notification settings - Fork 0
/
ChangeLog
executable file
·7364 lines (6370 loc) · 326 KB
/
ChangeLog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
2016-02-28 John Wu <John dot Wu at ACM dot org>
* NEWS: update for the new release 2.0.3
2016-01-31 John Wu <John dot Wu at ACM dot org>
* src/util.h: rename ibis::util::cleanDatasets to ibis::util::emptyCache
* src/part.cpp, src/query.cpp, src/filter.cpp: add calls to
ibis::util::emptyCache (previously, cleanDatasets)
2016-01-29 John Wu <John dot Wu at ACM dot org>
* src/util.cpp: in util::readDouble check the length of the string
before comparing against the constants
* src/part.cpp: throw exception rather than return a negative
error code after failing to allocate memory
* src/query.h, src/query.cpp: add function storeErrorMesg,
intending to record error message composed with ibis::util::logger
* src/util.h: add member functions str and c_str to
bis::util::logger class
2016-01-28 John Wu <John dot Wu at ACM dot org>
* examples/thula.cpp: add statements to print out the command line
arguments to aid debugging of trouble reported by check-sq2
* src/fileManager.cpp, src/bundle.cpp: change all message prefix
"Error" to "Warning"
* src/bundle.cpp: in bundles::sort check every column for the
minimum number of rows
2016-01-27 John Wu <John dot Wu at ACM dot org>
* win/FlexLexer.h: use the new version from macPorts for flex 3.0
* src/fromParser.yy, src/selectParser.yy, src/whereParser.yy:
require bison version 2.7 and newer, use define api.namespace (the
default flex and bison coming with Mac OS X is too old, the
versions from macPorts have deprecated the previous macros used)
* tests/readcsv.cpp: return -1 from readALine instead of 0
2016-01-26 John Wu <John dot Wu at ACM dot org>
* contrib/fbmerge/fbmerge.cpp: include <unistd.h> for getopt
* tests/Makefile.am: add flag -w to diff commands to ignore white
spaces in comparisons
* tests/readcsv.cpp: represent long integers as double or string
based on the number of digits (characters) used, previous version
was relative to sizeof(int)
* src/*: update copyright year
* tests/readcsv.cpp: attempt to skip line of spaces in readALine
2016-01-25 John Wu <John dot Wu at ACM dot org>
* src/util.h: remove throw statement from destructor of writeLock
* runconfig: update command to be used on MinGW (with correctly
installed gcc+autoconf+pthread)
* src/column.cpp, src/qExpr.cpp: do not attempt to use realtime
clock functions on windows platform (following the examples in
util.cpp)
* src/qExpr.cpp: in function math::toUnixTime::eval, replace
timegm with a more call to mktime
* src/fileManager.cpp: add conditional macro to limit the use of
getrlimit to unix systems
2016-01-23 John Wu <John dot Wu at ACM dot org>
* src/category.cpp:253 add writeLock per suggestion from Steven Enns
<saenns at a9 dot com>
2015-11-07 John Wu <John dot Wu at ACM dot org>
* examples/tiapi.c:281 change constant from 7 to 8 in the
expression to compute the number of expected values
2015-11-06 John Wu <John dot Wu at ACM dot org>
* src/part.cpp: change std::binder1st and std::binder2nd to
std::bind1st and std::bind2nd
2015-11-05 John Wu <John dot Wu at ACM dot org>
* src/part.cpp, src/column.cpp: invoke
ibis::fileManager::instance() to ensure the fileManager is
initialized
2015-11-04 John Wu <John dot Wu at ACM dot org>
* src/iapi.cpp: add documentation to clarify the operation of
fastbit_iapi_extend_array
2015-10-30 John Wu <John dot Wu at ACM dot org>
* src/category.cpp: comment out the code block in
ibis::text::startPositions that fill .sp file when the data file
is not present. This is not necessarily a useful thing to do as
pointed out by Steve Enns
* src/dictionary.cpp: update doxygen documentation to clarify the
merge function
* src/mensa.cpp: in function ibis::mensa::mergeCategories, add
calls to ibis::dictionary::sort to make sure the new dictionary
used is sorted
2015-10-22 John Wu <John dot Wu at ACM dot org>
* configure.ac: rearrange the test for java directory on mac to
accommodate the recent changes in OS X 10.11
2015-10-01 John Wu <John dot Wu at ACM dot org>
* src/column.cpp: in function column::purgeIndexFile comment out
the code that delete .dic and .int files, this will allow the
dictionaries supplied by users to be kept and will address the
problem reported by Jon Strabala on 9/10
* src/category.cpp: add code to adjust mask and write the new mask
to the corresponding mask file in text::startPositions, to add the
problem reported by masks with text values by Jon Strabala on 9/1
* part.h, part.cpp, bord.cpp, tafel.cpp: rename part::writeString
to be part::writeStrings, change its 1st argument to be a
filename, use fwrite to write the string to take advantage of the
system buffering. This should address the performance problem
observed by Jon Strabala (reported on 8/22)
2015-09-29 John Wu <John dot Wu at ACM dot org>
* src/*.h, src/*.cpp: rename stringList to stringArray, typeList
to typeArray, bufferList to bufferArray, to make it clear these
types are using ibis::array_t
* src/table.h: add type stringVector to create a new version of
ibis::table::parseNames
* src/fileManager.cpp: in getFile, surround tmp->enlarge call with
try-catch blocks to enable retry to allocate memory (but only
retry once)
* src/fileManager.cpp: in function fileManager::flushDir, correct
the error in the computation of offset (renamed to len1)
* examples/ibis.cpp: move dirs to be a global variable, invoke
flushDir in function clean_up
2015-09-28 John Wu <John dot Wu at ACM dot org>
* src/fileManager.h: comment out the code for softWriteLock, it is
not used
* src/fileManager.cpp: make sure there is enough space to read the
expected content before calling doRead, in an attempt to avoid
deadlock ibis::fileManager::mutex
* examples/ibis.cpp, examples/thula.cpp: replace the use of
ibis::table::stringList with std::vector<const char*> for local
variables -- these local variables were tiggering the invocation
of fileManager::instance() before the parameters for the file
manager is known
2015-09-12 John Wu <John dot Wu at ACM dot org>
* src/fileManager.h, src/fileManager.cpp: add function
getMaxOpenMmapFiles and getMaxMmapBytes per suggestion of Jon
Stabala
2015-09-12 John Wu <John dot Wu at ACM dot org>
* examples/ibis.cpp: change the number of test queries to be
determined by -t option instead of a product of number of threads
and the testing option, i.e., -t 100 will always generate 100
random queries. It will try to make sure each thread has two
queries to process, so when the number of threads is large, it
will still inscrease the number of random queries generated.
* test/Makefile.am: change check-sq0 and check-sq1 to use multiple
threads to speed up the test execution. On a MacBook Pro,
check-sq0 runs in 90 seconds.
* examples/ibis.cpp: change all invocations of buildIndexes to use
global variables indexingOptioins and build_index to control the
type of index built and the number of threads used
2015-09-02 John Wu <John dot Wu at ACM dot org>
* src/fileManger.cpp:634 move tests on OPEN_MAX and STREAM_MAX as
fallback option of sysconf(_SC_OPEN_MAX), which should give the
correct number of file descriptors available at runtime, the other
macros contains static file descriptor limits predefined by POSIX
standards (to address a problem observed by Jon Stabala)
2015-08-28 John Wu <John dot Wu at ACM dot org>
* examples/ibis.cpp: minor reordering of states that read
configuration files supplied from the command line
2015-08-23 John Wu <John dot Wu at ACM dot org>
* src/utilidor.cpp: protect all calls to strcmp to prevent sending
null pointer to it (strcmp(A, B) > 0 ==> (A != 0 && (B == 0 ||
strcmp(A, B) > 0)), strcmp(A, B) >= 0 ==> ((A != 0 && (B == 0 ||
strcmp(A, B) >= 0)) || (A == 0 && B == 0))
* tests/Makefile.am: adapted changes from Jon Strabala to
tests/Makefile.in to tests/Makefile.am to address problems with
more strict sh script syntax under solaris derivatives (works on
Mac OS X 10.10.5, will ask Jon to try the new Makefile.am)
2015-08-19 John Wu <John dot Wu at ACM dot org>
* src/ibis.h: update the in-line documentation of
ibis::index::INDEX_TYPE
2015-06-20 John Wu <John dot Wu at ACM dot org>
* src/query.cpp, src/qExpr.cpp: fix problem associated with
estimation of query processing costs, a problem behind the use
case reported by Petr Velan on 5/7/2015
2015-06-14 John Wu <John dot Wu at ACM dot org>
* src/index.cpp: add timer to all versions of index::activate at
verboseness level 4
2015-05-08 John Wu <John dot Wu at ACM dot org>
* examples/thula.cpp: update operator "distinct" to
"countdistinct"
2015-05-01 John Wu <John dot Wu at ACM dot org>
* src/util.h: change the read functions of sharedInt32 and
sharedInt64 to invoke the mutex lock when atomic is not supported
(per request of Jilong Wang)
* src/util.h, configure.ac: add conditional macro for std::atomic,
use std::atomic in sharedInt32 and sharedInt64 when
HAVE_ATOMIC_TEMPLATE
2015-04-30 John Wu <John dot Wu at ACM dot org>
* src/part.cpp: in function getColumn, the column name of the form
'__xx' is interpreted as decimal numbers, but the function
selectClause::fillNames that generated these names are producing
hexadecimal numbers
2015-03-31 Petr Velan <petr dot velan at cesnet dot cz>
* src/util.cpp: move the include <pwd.h> statement to after util.h
is included because HAVE_GETPWUID is defined in util.h (which
eventually get it from src/fastbit-config.h)
2015-03-30 John Wu <John dot Wu at ACM dot org>
* configure.ac: add -DWITHOUT_FASTBIT_CONFIG_H when attempting to
compile util.cpp for testing the compiler flags needed for popen
and pclose
2015-03-25 John Wu <John dot Wu at ACM dot org>
* src/const.h: on MINGW, stricmp should be _stricmp
* src/util.cpp: add conditional compilation for pwd.h on MinGW
2015-03-24 John Wu <John dot Wu at ACM dot org>
* runconfig: add special flags for compiling on cygwin and mingw
2015-03-21 John Wu <John dot Wu at ACM dot org>
* src/part.cpp: countPages return at least one when the mask is
not empty (this could happen when all the elements are on the same
page)
2015-03-18 John Wu <John dot Wu at ACM dot org>
* win: convert *.vcproj to *.vcxproj, update ibis.sln, rename the
old file as ibis-vc8.sln
2015-03-11 John Wu <John dot Wu at ACM dot org>
* src/iapi.cpp: add print statements to public functions
2015-03-04 John Wu <John dot Wu at ACM dot org>
* src/colValues.cpp: review all sorting functions to check for the
condition identified by Divya
2015-03-04 Divya Chandrasekar <chadivya at a9 dot com>
* src/colValues.cpp: add statement to sort the second half of the
list of strings
2015-03-02 John Wu <John dot Wu at ACM dot org>
* example/tiapi.c:476 logical error, "nv=n;" should have been
"nv=n+n;"
* src/iapi.h: add functions
fastbit_iapi_register_selection_as_bit_array and
fastbit_iapi_extend_bit_array_with_selection
2015-02-28 John Wu <John dot Wu at ACM dot org>
* src/bitvector.h:781 nbits should have been active.nbits
2015-02-26 John Wu <John dot Wu at ACM dot org>
* bitvector.cpp:602 ns should be nb - this causes bad bug in the
indexes
2015-02-25 Justin Swanhart <greenlion at gmail dot com>
* src/tafel.cpp: patch writeData to correctly print the number of
rows handles
* src/ibis.cpp: change reorder function to walk through all
subdirectories
2015-02-24 John Wu <John dot Wu at ACM dot org>
* src/bord.cpp: prevent column::append to work with index only
column, in copy constructor, copy index as well as data values
* examples/tiapi.c: move the code that makes a3 index only in
queryarray2 to after the doubling of data
2015-02-22 John Wu <John dot Wu at ACM dot org>
* src/bord.cpp: update bord::column::append to handle column type
BIT following the new storage scheme
2015-02-21 John Wu <John dot Wu at ACM dot org>
* src/part.cpp:19568 "cols[i] == 0" should be "cols[i] != 0"
* src/bord.cpp: change back the representation of BIT to separate
the content of buffer from mask_ -- to address the problem of a BIT
2015-02-20 John Wu <John dot Wu at ACM dot org>
* src/iapi.cpp: modify functions that frees an array to reduce the
size of the global std::vector used to keep track of all arrays
2015-02-17 John Wu <John dot Wu at ACM dot org>
* src/bord.cpp: update to work with type BIT in the constructors
of ibis::bord::column
2015-02-10 John Wu <John dot Wu at ACM dot org>
* src/table.h, src/column.cpp: remove the unused global variable
TYPECODE (avoid the need to update it for the addition of BIT
type)
* src/bitvector.h, src/bitvector64.h: add function appendByte
* src/bord.cpp: change ibis::bord::column to use mask_ for columns
with 0/1 values (i.e., bit values)
* src/iapi.h, src/iapi.cpp: add function fastbit_iapi_extend_array
2015-02-09 John Wu <John dot Wu at ACM dot org>
* src/capi.h, src/bord.h, src/bord.cpp: add type ibis::BIT
2015-02-06 John Wu <John dot Wu at ACM dot org>
* src/capi.h, src/capi.cpp: add function fastbit_read_clock to
report the current time in seconds
2015-02-05 John Wu <John dot Wu at ACM dot org>
* runconfig: change the default path of FastBit install directory
to not include fastbit
* tests/scripts/fastbit-config.in: move notes into usage
2015-01-31 John Wu <John dot Wu at ACM dot org>
* src/category.cpp: Chen's change is equivalent to
text::readStrings, rename the original selectStrings to
text::readStrings2 and the original text::readStrings to
text::readStrings1, add new implementation of selectStrings in
terms of readStrings1 and readStrings2
2015-01-30 Chen HANG <hangchen at diditaxi dot com dot cn>
* src/category.cpp: change text::selectStrings to read values from
.sp file instead of using a file map. This improves performance
when one or two entries are read from a text file
2015-01-28 John Wu <John dot Wu at ACM dot org>
* src/table.h, src/mensa.cpp: add functions
ibis::table::isValidName and ibis::table::consecrateName
2015-01-26 John Wu <John dot Wu at ACM dot org>
* src/part.cpp: check for the integers read back from metadata
file in function readMetaData
2015-01-25 John Wu <John dot Wu at ACM dot org>
* src/iapi.cpp: correct the misuses of the declarations of
mutexLock
2015-01-23 John Wu <John dot Wu at ACM dot org>
* src/column.h, src/iapi.h: add notes about the names of the
columns
2015-01-04 John Wu <John dot Wu at ACM dot org>
* java/Makefile.am: change include_HEADERS to pkginclude_HEADERS to
allow installing in standard system directory
* tests/scripts/fastbit-config.in: add notes about including
FastBit header files as <fastbit/ibis.h> instead of <ibis.h>
2014-01-04 Martin Hauke <mardnh at gmx.de>
* src/Makefile.am: change include_HEADERS to pkginclude_HEADERS to
allow installing in standard system directory
2014-12-28 John Wu <John dot Wu at ACM dot org>
* tests/acmpub.v.a, tests/acmpub.v.b: modify the test dictionary
to exercise the new feature in dictionary class
* src/bord.cpp: update the use of dictionaries to allow 0 as a
regular code word
2014-12-27 John Wu <John dot Wu at ACM dot org>
* src/ikeywords.cpp: update functions to reflect the change in
dictionary
2014-12-26 John Wu <John dot Wu at ACM dot org>
* src/dictionary.h, src/dictionary.cpp: modify the dictionary code
to allow users to pass in an arbitrary combination of integers and
strings, change the integer code for null values from 0 to -1
(0xFFFFFFFFU in 32-bit unsigned integer format)
2014-12-16 John Wu <John dot Wu at ACM dot org>
* src/bitvector.cpp: change do_cnt() to use local variable to
count the number of set bits so as to avoid the thread-safety
issue, define FASTBIT_LAZY_INIT to be 1 (i.e., go back to lazy
initialization in bitvectors, reversing the change makde on
2014/08/06)
2014-12-15 John Wu <John dot Wu at ACM dot org>
* src/bitvector.cpp: in do_cnt() add code blocks to reduce the
future cost of counting a bitvector of all 0s
* src/query.cpp: in doEvaluate remove the test on ht.cnt() to
avoid a problem reported by Steven Enns
2014-12-14 John Wu <John dot Wu at ACM dot org>
* configure.ac: add test for presence of strptime
* src/whereParser.yy: add conditional macro HAVE_STRPTIME to check
for the presence of strptime before using it. If strptime is not
available, TO_UNIXTIME_LOCAL and TO_UNIXTIME_GMT will cause
exceptions
2014-12-11 John Wu <John dot Wu at ACM dot org>
* INSTALL: update to reflect the recent failed tests with intel,
PGI and pathscale compilers on hopper.nersc.gov. These compilers
do not have support for either unique_ptr or unordered_map.
2014-12-07 John Wu <John dot Wu at ACM dot org>
* src/column.cpp:5641,5703 remove the check of noffsets > nkeys,
to allow binned index with a single bitmap to be accepted (to
address a problem reported by Junmin Gu)
2014-12-06 John Wu <John dot Wu at ACM dot org>
* examples/tiapi.c: add fillarray2 and queryarray2 to exercise the
case of indexing values that are all the same in an attempt to
address the problem reported by Junmin Gu
2014-11-26 John Wu <John dot Wu at ACM dot org>
* examples/ardea.cpp: invoke tablex::setASCIIDictionary to make
use of the user supplied dictionary file
* tests/Makefile.am: add test cases 13, 14 and 15 to exercise the
text data with external ASCII dictionaries
2014-11-24 John Wu <John dot Wu at ACM dot org>
* doc/footer.html: fix path to cowboy1a.gif
* examples/thula.cpp: remove -q option, add explanation of -x
option, per suggestion from Jon Strabala
* examples/ibis.cpp: streamline output option to ensure the
existing content of the output file is cleared before running any
queries, this option leaves the output file empty when query
operations encounter any error (suggested by Jon Strabala)
2014-11-19 John Wu <John dot Wu at ACM dot org>
* tests/Makefile.am: add test cases based on input from Jon
Strabala
2014-11-10 Jon Strabala <jon dot strabala at quantumsi dot com>
* src/whereParser.yy: addressing the issue with tm_mday being left
as 0 by the conversion functions
2014-11-07 John Wu <John dot Wu at ACM dot org>
* COPYING: replace the license with the 3-clause BSD
* configure.ac: mark the current soruce code as 2.0.0
2014-11-03 John Wu <John dot Wu at ACM dot org>
* src/ifade.cpp, src/ixpack.cpp, src/ixpale.cpp, src/ixzone.cpp:
add statement to log conditions leading to error code -13 to
debug a problem reported by Fasika Daksa
* src/whereParser.yy:1195 initialize tm_isdst to -1 to tell mktime
to figure out whether DST is in effect (a solution found by Jon
Strabala)
* test/Makefile.am: add test case check-js2 based on examples from
Jon Strabala
2014-11-01 John Wu <John dot Wu at ACM dot org>
* src/whereParser.yy: update TO_UNIXTIME_LOCAL and TO_UNIXTIME_GMT
based on suggestion from Jon Strabala
* src/table.h, src/tafel.h, src/tafel.cpp: add functions
getASCIIDictionary and setASCIIDictionary
2014-10-29 John Wu <John dot Wu at ACM dot org>
* src/whereLexer.ll, src/whereParser.yy: change TO_UNIXTIME_LOCAL
and TO_UNIXTIME_GMT to be the inverse of FROM_UNIXTIME_LOCAL and
FROM_UNIXTIME_GMT
* src/dictioinary.h, src/dictionary.cpp: rename print to toASCII
and add function fromASCII to read the output of toASCII
2014-10-28 John Wu <John dot Wu at ACM dot org>
* src/qExpr.h: assign the left child in the copy constructors of
customFunction1 and stringFunction1 to avoid the problem seg fault
reported by Jon Strabala when using TO_UNIXTIME_LOCAL
2014-10-25 John Wu <John dot Wu at ACM dot org>
* src/qExpr.h, src/qExpr.cpp: add stringFunction1
* src/selectParser.yy: change FORMAT_UNIXTIME_LOCAL and
FORMAT_UNIXTIME_GMT to use the new stringFunction1 instead of
treating them directives for printing only. This change should
make these function functions behave like existing functions and
address the consistency issues raised by Jon Strabala.
* src/part.h, src/part.cpp: add new vresion of function calculate
to evaluate the string value functions
2014-10-24 John Wu <John dot Wu at ACM dot org>
* src/*.cpp: change varaibles receiving return values of UnixWrite
from int to off_t per suggestion of Zheng Yan from Harvard
* src/colValues.cpp: sorting functions could leave the 2nd element
of the array in the incorrect order (a problem reported by Petr
Velan)
2014-10-23 John Wu <John dot Wu at ACM dot org>
* src/selectParser.yy, src/selectLexer.ll: turn
FORMAT_UNIXTIME_LOCAL and FORMAT_UNIXTIME_GMT into fixed tokens to
avoid conflicts with two-argument functions
* Makefile.am: put back bin_SCRIPTS line to install fastbit-config
script (a problem pointed out by Gunther Weber)
2014-10-22 John Wu <John dot Wu at ACM dot org>
* src/selectParser.yy: add warning message about ignoring
formating directive on unix time stamps (following suggestion from
Jon Strabala)
2014-10-21 John Wu <John dot Wu at ACM dot org>
* src/qExpr.h, src/qExpr.cpp, src/whereLexer.ll,
src/whereParser.yy: add an initial implementation of functions in
where clause: from_unixtime_local, from_unixtime_gmt,
to_unixtime_local, to_unixtime_gmt (based on discussion with Jon
Strabala at QuantumSI dot com)
* src/whereClause.h: add explanation of the four time handling
functions.
2014-10-20 John Wu <John dot Wu at ACM dot org>
* src/qExpr.h, src/qExpr.cpp, src/column.h, src/column.cpp: rename
format_unixtime to unixTimeScribe
* src/colValues.h, src/colValues.cpp: add struct
ibis::colValues::format_unixtime to allow a column integer or
floating-point values to be printed out as unix time stamps
2014-10-18 John Wu <John dot Wu at ACM dot org>
* doc/ibisCommandLine.html: update document to reflect the currect
ibis code
* tests/scripts/star2002.sh: add script to the list of files
distributed (the previous dataset used to produce the sample
output in doc/ibisCommandLine.html was not captured and can not
reproduced)
* doc/*.html: change all html page to use https instead of http to
avoid complains about mixed content
* examples/ibis.cpp: add option -m to allow mesh queries to by
pass tableSelect (pass make check)
2014-10-17 John Wu <John dot Wu at ACM dot org>
* examples/ibis.cpp: add global variable independent_parts to
indicate whether to process each data partition separately. By
default use tableSelect to treat all data partitions as a single
data table, which should produce consistent results for the test
cases from Jon Strabala
2014-10-16 John Wu <John dot Wu at ACM dot org>
* src/util.h, src/dictionary.h: move the specialization of
std::hash from util.h to dictionary.h to limit the files that
needs to include header file hash_map
2014-10-13 John Wu <John dot Wu at ACM dot org>
* configure.ac: extend the checking of std::unordered_map to avoid
problem of unset variable
* src/dictionary.h: add option to use __gnu_cxx::hash_map instead
of std::unordered_map to accommodate order compilers
2014-10-05 John Wu <John dot Wu at ACM dot org>
* src/selectClause.cpp:534 nexted to output numbers in hex format
(which creates inconsistent internal names for referring to
expressions without aliases)
2014-10-02 John Wu <John dot Wu at ACM dot org>
* src/iapi.cpp:1742 remove bound == 0 (an error discovered by
Junmin Gu)
2014-09-20 John Wu <John dot Wu at ACM dot org>
* src/index.cpp: when activate a set of bitvectors between i and
j, the offsets for buf was computed incorrectly, which leads to
wrong bitvectors being reconstructed
2014-09-17 John Wu <John dot Wu at ACM dot org>
* src/index.h: add a new variant of index::bitmapReader::read that
takes array_t<uint32_t>
* src/bitvector.h: add new constructor of bitvector that uses a
part of an array_t object
* src/index.cpp: use array_t<uint32_t> with index::bitmapReader so
that temporary storage could be reference counted, to address the
problem reported by Jerome Soumagne
2014-09-11 John Wu <John dot Wu at ACM dot org>
* src/column.cpp: in function attachIndex, set the mask to all 1s
if the existing mask is empty
* src/bord.cpp: in constructor of bord from a list of columns, ask
each column for the number of rows
* src/bord.h: add function ibis::bord::column::getDataflag and
ibis::bord::column::setDataflag
2014-09-10 John Wu <John dot Wu at ACM dot org>
* src/bord.cpp: in constructor of bord, check the validify of
pointer buf before using (to address a problem reported by Jerome
Soumagne)
2014-09-03 John Wu <John dot Wu at ACM dot org>
* src/category.cpp: in ibis::text::stringSearch, adjust the LOGGER
statements to prevent the end of file being recognied as an error
(a problem reported by Marek mdmarek at gmail.com)
* src/util.h: add ibis::util::flock to wrap around posix function
flock, invoke util::flock to make sure an exclusive lock on the
index is acquired before attempting to write the actual content of
an index. This is intended to address the problem reported by
Petr Velan.
2014-09-02 John Wu <John dot Wu at ACM dot org>
* src/tafel.cpp: clarify the inline documentation on type handling
in parseNamesAndTypes
2014-08-21 John Wu <John dot Wu at ACM dot org>
* configure.ac:587 add case for darwin per suggestion from mdmarek
at gmail.com
2014-08-12 John Wu <John dot Wu at ACM dot org>
* src/tafel.cpp:3991 specify the size of the substring to compare
prefix "unsigned" (a problem reported by Li Jun
<lijunli2598@gmail.com>), add code to deal with arbitrary number
of spaces after "unsigned"
2014-08-06 John Wu <John dot Wu at ACM dot org>
* src/bitvector.cpp, src/bitvector64.cpp: change constructor to
compute the sizes of bitvector and bitvector64 during construction
to avoid thread-safety issue with do_cnt, a problem reported by
Dominique Prunier
2014-08-02 John Wu <John dot Wu at ACM dot org>
* src/fastbit-config.h.in: add HAVE_SNPRINTF, change all uses of
snprintf to check HAVE_SNPRINTF
2014-07-29 John Wu <John dot Wu at ACM dot org>
* src/capi.cpp: in fastbit_part_list::find, return a nil pointer
if the existing data partition is not readable. This
re-examination was prompted by an error report from Mike Carter.
* src/dictionary.cpp: Mike Carter indentified a case of support
for unordered_map without member function reserve, remove the call
to reserve in readKeys0
2014-07-28 John Wu <John dot Wu at ACM dot org>
* src/util.h, src/part.cpp: add function ibis::util::cleanDatasets
* src/query.cpp: in case of query evaluation failure, attempt to
clean all datasets (by calling ibis::util::cleanDatasets) before
try again
2014-07-27 John Wu <John dot Wu at ACM dot org>
* src/part.h: add function clear to clear the existing content of
a data partition if it is not in use (based on softWriteLock)
* src/capi.cpp: change fastbit_part_list::clear to be a public
function that invokes ibis::part::clear to remove the content
before deleting the object
* src/category.cpp: move mutex lock to cover the entire body of
function category::prepareMembers
2014-07-26 John Wu <John dot Wu at ACM dot org>
* src/parti.cpp: in function updateData, call emptyCache after the
time stamp has been checked to avoid unnecessary changes and to
protect the call with the write lock
* src/capi.cpp: rearrange the mutex lock to better protect the
shared/global variable _capi_tlist and _capi_tablex in capi.cpp --
this addresses an issue of repeated creation and deletion of
gov.lbl.fastbit.FastBit eventually leading to a seg fault because
of the race condition between the garbage collector removing
_capi_tlist and the other functions trying to use the same pointer
* java/Makefile.am: add rules to create and install fastbitjni.jar
into the library directory
2014-07-24 John Wu <John dot Wu at ACM dot org>
* configure.ac: add subdirectory darwin to the list of include
directories to combat problem of jni_md.h
2014-07-23 John Wu <John dot Wu at ACM dot org>
* examples/tiapi.c:48 move declaration of variable j outside of
the for loop to conform to C compiler requirement
2014-07-22 John Wu <John dot Wu at ACM dot org>
* src/column.h, src/column.cpp: remove the check for nil data file
name from column::selectValues because the underlying
implementation actually deal with the case correctly previously,
the recent changes were incorrect
2014-07-20 John Wu <John dot Wu at ACM dot org>
* src/column.h, src/column.cpp: add member variable dataflag
2014-07-18 John Wu <John dot Wu at ACM dot org>
* examples/tiapi.c: update to work with index only data column
2014-07-17 John Wu <John dot Wu at ACM dot org>
* src/bord.h, src/bord.cpp: add new constructor to
ibis::bord::column to accept data through FastBitReadExtArray
* src/column.h: rename writeIndex to indexWrite, rename
serialSizes to indexSerialSizes
2014-07-16 John Wu <John dot Wu at ACM dot org>
* src/const.h: rename FastBitReadIntArray to FastBitReadBitmaps
2014-07-15 John Wu <John dot Wu at ACM dot org>
* src/const.h: add FastBitReadExtArray as the generic function
propotype for reading external data
2014-07-07 John Wu <John dot Wu at ACM dot org>
* src/iapi.h: clearify the fact that the bitmap keys and offsets
are used by FastBit data structures and therefore can not be freed
by the caller until fastbit_iapi_free_all is called -- fixing a
logical error in tiapi.c related to freeing keys and offsets
prematurely
2014-06-29 John Wu <John dot Wu at ACM dot org>
* src/ibin.h, src/irelic.h: add column object to the new
constructors for ibis::bin and ibis::relic in order to set nrows
* src/column.cpp: update attachIndex to use the new construtors
* src/index.cpp: in the copy constructor, need to copy each
bitvector in bits
2014-06-28 John Wu <John dot Wu at ACM dot org>
* src/index.h: add virtual function dup to ibis::index class
hierarchy, add copy constructors
* src/column.cpp: in the copy constructor of ibis::column, invoke
ibis::index::dup to copy the index when present
* src/column.h: add function ibis::column::nRows()
* src/index.cpp: add statements to set nrows in initBitmaps
2014-06-27 John Wu <John dot Wu at ACM dot org>
* src/index.cpp: rename bheadle to breader, update versions of
activate to make use breader for recreating the bitvectors
* src/column.h, src/column.cpp: add function attachIndex for
associating and index with a column
* src/iapi.h, src/iapi.cpp: add fastbit_selection_osr to replace
fastbit_selection_create, add fastbit_iapi_attach_index to
associate an index with an array
* src/ibin.cpp:6016 check for nobs==1 to prevent array out of
bounds problems
* src/iapi.cpp: revise implementation of
fastbit_iapi_deconstruct_index to explicit copy the values from
array_t objects to the output buffers
* src/iapi.cpp:1491 _4 was in the wrong place
* src/fileManager.h, src/array_t.h: add function release to give
the space to user as a raw pointer.
* src/iapi.h: remove the size information from
fastbit_iapi_build_index, change fastbit_iapi_deconstruct_index to
allocate the required memory
* example/tiapi.c: modify the example to use the callback function
FastBitReadIntArray to avoid passing all bitmaps to construct the
index
2014-06-26 John Wu <John dot Wu at ACM dot org>
* src/iapi.h, src/iapi.cpp: add functions to deal with indexes
* example/ibis.cpp: increase gVerbose when using
ibis::bundle::print to generate output to files (to keep the
regression suite running error free)
* src/const.h: add function type FastBitReadIntArray
* src/index.h: add class ibis::index::bitmapReader based on
FastBitReadIntArray, add a new version of initBitmaps that accept
FastBitReadIntArray (and initializes a bitmapReader object)
* src/irelic.h, src/ibin.h: add constructors that makes use of the
new initBitmaps
* src/column.cpp: modify loadIndex to not quit when thePart is nil
* src/irelic.cpp, src/ibin.cpp: modify the construct to work with
in-memory data, as opposed to requiring explicit invocation of the
special construction function
2014-06-25 John Wu <John dot Wu at ACM dot org>
* src/index.h: add function serialSizes to capture the sizes of
the three arrays to be generated by the function write. This is
necessary for the IAPI to give users a chance to allocate the
correct sizes for the output buffers
2014-06-19 John Wu <John dot Wu at ACM dot org>
* src/bundles.cpp: limit the number of rows printed from
bundles::print - in an attempt to understand the out-of-memory
fault reported by Alan Kemp <alan at irisns.com>
* src/bord.cpp: limit the number of rows printed from bord::merge
* src/bordm.cpp: add in-line documentation to bord::limit and
bord::column::limit, change column::limit to invoke nosharing only
if the content has been modified
2014-06-17 John Wu <John dot Wu at ACM dot org>
* src/bundle.cpp:1466 rewrite the conditional test to capture the
case where the name cn do not match c->name() -- to address a
problem reported by Hendrik Heinemann <hendrik.heinemann at
questback.com>
2014-06-15 John Wu <John dot Wu at ACM dot org>
* src/bundles.cpp:1500 move the LOGGER statement before push_back
2014-05-22 John Wu <John dot Wu at ACM dot org>
* autoconfig.ac: remove the prefix 'ibis' from the version number
based on a recommendation from lscott@kiemaadvisors.com
2014-05-16 Zhongbo Tian <windreamer at gmail dot com>
* src/mensa.cpp: fix the order of bound check in
ibis::mensa::cursor::fetch
2014-05-15 John Wu <John dot Wu at ACM dot org>
* src/mensa.cpp: correct the problems with getColumnAsTTTs (a
problem identified by Zhongbo Tian
* src/bord.cpp: switch std::copy to memcpy when the types match in
getColumnAsTTTs
* src/selectClause.cpp:197 remove the comparison involving aggr_
to address a problem reported by Bernd Jaenichen
2014-05-14 John Wu <John dot Wu at ACM dot org>
* src/countQuery.cpp:425, src/query.cpp:2732 (high.sloppyCount() >
0) should be (low.sloppyCount() > 0) (an error discovered by a
test case from Junmin Gu)
2014-05-11 John Wu <John dot Wu at ACM dot org>
* src/ibin.cpp: modify bin::print to always print the last bin
2014-05-10 John Wu <John dot Wu at ACM dot org>
* src/index.h: move printHeader to be a public static member
function
* src/ibin.cpp: check whether col is nil before use
* src/column.h: move template functions for computing min and max
to be static member functions
2014-05-09 John Wu <John dot Wu at ACM dot org>
* src/index.cpp: change index::create to separate out the portion
about reading an existing index, allow c=0 in this case
2014-04-29 John Wu <John dot Wu at ACM dot org>
* src/ibin.cpp: in bin::parsePrec, change the default precision to
2 digits; in constructors of ibis::bin, change index spec of nil
pointer, empty string, "automatic" or "default" to invoke
mapGranule, which effectively means <binning precision=2/>
2014-04-14 John Wu <John dot Wu at ACM dot org>
* src/index.cpp: regenerate empty bitmaps when reading from a file
(the changes on 4/10 neglected some branches of the activate functions)
2014-04-11 John Wu <John dot Wu at ACM dot org>
* src/array_t.cpp: in function reserve, add call to memset to put
0 in the newly allocated memory
* configure.ac: add --with-debug=d to control -DDEBUG option in
Makefiles (per discussion with Alex Romosan)
2014-04-10 John Wu <John dot Wu at ACM dot org>
* examples/tiapi.c: add initialization for nmax and msglvl
* src/index.cpp: regenerate an in-memory bitvector (of all 0s)
when the index file does not record anything for the bitvector
2014-04-10 Zhongbo Tian <windreamer at gmail dot com>
* src/bord.cpp: add code to handle string valued column
(in-memory) without a dictionary
* src/mensa.cpp: remove call to getValuesArray on string-valued
columns
2014-04-08 John Wu <John dot Wu at ACM dot org>
* src/iapi.h, src/iapi.cpp: rename the public functions of IAPI to
be more consistent, fix a couple of obvious bugs in iapi.cpp
2014-04-07 John Wu <John dot Wu at ACM dot org>
* runconfig: add entry for edison, determined that intel CC can
not support std::unique_ptr, have to use GNU CC
2014-03-30 John Wu <John dot Wu at ACM dot org>
* src/index.cpp: reverse the change on 3/24, did not spot that
fname was deleted once already, the new statement creates a double
deletion problem
2014-03-24 John Wu <John dot Wu at ACM dot org>
* src/index.cpp:1259 add statement to delete fname (all existing
indexing classes allocate space for it through
ibis::util::strnewdup)
2014-03-19 John Wu <John dot Wu at ACM dot org>
* src/dictionary.cpp: add timer to ibis::dictionary::write to
mirror the timer in ibis::dictionary::read
2014-03-17 Sean McNamara <Sean.McNamara at Webtrends dot com>
* configure.ac: fix problem with AC_MSG_ERROR, add test to try
c++0x when c++11 is not available
2014-03-13 John Wu <John dot Wu at ACM dot org>
* src/idirekte.cpp:117 set all bits of bits[popu] to 1s, to avoid
invoking ibis::column::getNullMask which needs a mutex lock.
Addressing the mutex lock problem reported by Sean McNamara
2014-03-13 John Wu <John dot Wu at ACM dot org>
* selectClause.h: minor changes to address warnings from doxygen
2014-03-07 John Wu <John dot Wu at ACM dot org>
* src/dictionary.cpp:421 ith string in the alphabetic order should
be recorded as string ik in raw_
2014-03-04 John Wu <John dot Wu at ACM dot org>
* src/bitvector.h: add ibis::bitvector::pit
2014-02-21 John Wu <John dot Wu at ACM dot org>
* src/bord.cpp:214 neglected to set amask
2014-02-20 John Wu <John dot Wu at ACM dot org>
* src/column.cpp: modify versions of column::searchSorted to make
use of the function getValuesArray to work with in-memory data
* src/iroster.cpp: in roster::icSort make use of
column::getValuesArray when column::dataFileName can not generate
a valid file name
* src/qExpr.cpp: change versions of function overlap to return
true when the two arguments do not form a valid range (the ranges
may be invalid when the lower and upper bounds of a column has not
been set)
2014-02-19 John Wu <John dot Wu at ACM dot org>
* src/iapi.h, src/iapi.cpp: add functions to work with coordinates
* src/bord.cpp: make a copy of columns from the user provided list
of columns, becaue the destructor of ibis::part delete the columns
it knows about
* examples/tiapi.c: add tester for IAPI
2014-02-18 John Wu <John dot Wu at ACM dot org>
* src/bord.h: add member variable shape to ibis::bord::column, add
a new constructor
2014-02-17 John Wu <John dot Wu at ACM dot org>
* src/iapi.h, src/iapi.cpp: add a new in-memory API that
constructs query expressions through function calls (without a
parser of strings)
2014-02-16 John Wu <John dot Wu at ACM dot org>
* src/bord.h, src/bord.cpp: add constructor to take a list of
columns as input
2014-02-12 John Wu <John dot Wu at ACM dot org>
* src/column.cpp: check argument 1 (col) in the constructor of
ibis::column::indexLock, do nothing when col == 0 (this allows a
nil column object to be used to create an index lock, however,
that index lock will do nothing)
2014-02-11 John Wu <John dot Wu at ACM dot org>
* src/index.cpp, src/idirekte.cpp: check for col being nil
2014-02-10 John Wu <John dot Wu at ACM dot org>
* src/selectClause.cpp:397 change <= to <, the equal sign allows
a selection expression without an aggregator to be declared as
being separable, which violates the intention of the function
(this addresses the problem reported by Sean McNamara on 2/7/2014)
2014-02-06 John Wu <John dot Wu at ACM dot org>
* src/column.h, src/column.cpp: add function fullname to deal with
the printing of column names
* src/irelic.cpp, src/ibin.cpp: allow the read function and some
constructors to proceed with null col value
* src/column.h: change column::mutexLock to allow the first
argument of the constructor to be nil
* src/index.cpp: change versions of function activate to allow the
member variable col to be nil
2014-02-05 John Wu <John dot Wu at ACM dot org>
* src/dictionary.cpp: clean up the function mergeBuffers
2014-02-04 John Wu <John dot Wu at ACM dot org>
* configure.ac: change uint32_t to unsigned in the sample test for
c++11 flag
* src/dictionary.h, src/dictionary.cpp: add function mergeBuffers
and make use of mergeBuffers in function write to reduce I/O time
2014-02-03 John Wu <John dot Wu at ACM dot org>
* src/dictionary.h: add function dictionary::print
* src/dictionary.cpp: function readKeys0 has an off-by-1 problem
that causes it to read the previous version of dictionary
incorrectly (with integer 1 left undefined, found in debugging a
test case reported by Alex Romosan)
2014-02-02 John Wu <John dot Wu at ACM dot org>
* configure.ac: modify the block that try to compile src/util.cpp
to use a single statement on std::unordered_map (to minimize
dependencies in util.h, a problem reported by Alex Romosan)
2014-01-27 John Wu <John dot Wu at ACM dot org>
* tests/scripts/fastbit-config.in: add option --cxxextra to
display the extra option required to compile C++ code
2014-01-24 John Wu <John dot Wu at ACM dot org>
* configure.ac: add statements to check if --std=c++11 is needed
(to remove the need to pass extra flags from command line)
* src/index.cpp: allow .idx filename to be used directly without
modification (promised in the inline documentation, but not
implemented)
* src/tests/scripts/fastbit-config: add option -cxxflags
* src/dictionary.h, src/dictionary.cpp: change the offsets to be
64-bit long in dictionary files, introduce new version tag for the
dictionary files, add support for reading previous versions of
dictionary files
2014-01-22 John Wu <John dot Wu at ACM dot org>
* src/dictionary.h: a new implementation based on
std::unordered_map with murmurhash for strings
2014-01-21 John Wu <John dot Wu at ACM dot org>
* src/utilidor.cpp:1598 keys[i] should be ktmp, otherwise the
arrays are not sorted (a serious flaw)
2014-01-20 John Wu <John dot Wu at ACM dot org>
* src/dictionary.h, src/dictionary.cpp: implement the dictionary
based on unordered_map (which requires -std=c++11)
* src/utilidor.cpp: rename sortStrings_quick to sortStrings, and
make the function visible outside of utilidor.cpp (to be used on
dictionary.cpp:554 in the function dictionary::sort)
2014-01-19 John Wu <John dot Wu at ACM dot org>
* src/*.h, src/*.cpp, examples/*.cpp: replace include file
string.h with cstring, replace auto_ptr with unqiue_ptr
2014-01-17 John Wu <John dot Wu at ACM dot org>
* src/dictionary.h, src/dictionary.cpp: add function appendOrdered
to work with strings that are already sorted
2014-01-11 John Wu <John dot Wu at ACM dot org>
* src/ibin.cpp:4863 modify the last bin boundary if it is less
than or equal to the nominal upper bound
* src/part.cpp: print the queries out from buildQueryList when the
verboseness level is 5 or more
* src/column.cpp:5767 add statement to assign value to ierr before
returning (fix the problem revealed by make full-check)
2014-01-10 John Wu <John dot Wu at ACM dot org>
* src/index.cpp:876 only checked the 1st character of the index
specificiation, add code to skip all leading blank spaces
2014-01-08 John Wu <John dot Wu at ACM dot org>
* src/tafel.cpp:2450 change the type of tmp from uint32_t to
size_t to accomodate large memory buffers
2014-01-07 John Wu <John dot Wu at ACM dot org>
* src/tafel.h, src/tafel.cpp: rename reserveSpace to reserveBuffer
* tests/jrf.cpp: use double to capture the return value of
fileManager::currentCacheSize, previous version of using 32-bit
integers which overflows on a machine with more than 4GB of memory
2013-12-28 John Wu <John dot Wu at ACM dot org>
* src/resource.h, src/resource.cpp: modify function resource::read
to return an error code instead of printing out warning messages
because the mechanism for printing message could invoke to
function ibis::gParameter which causes recursive calls to a static
constructor in ibis::gParameter
* src/resource.cpp: in the default constructor and
ibis::gParameters, avoid attempt to read the default configuration
files. Instead, change the documentation to require the user to