-
Notifications
You must be signed in to change notification settings - Fork 22
/
NOTES.txt
3261 lines (2523 loc) · 122 KB
/
NOTES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Increase/bump the release version:
----------------------------------
- all below is outdated, everything is handled by the tools/deploy.ps1 script
- Gentoo tinyorm ebuild isn't currently part of the tools/deploy.ps1, create a new ebuild before
executing the tools/deploy.ps1 script ❗
- bump message format:
bump version to TinyORM v0.37.3 and tom v0.9.1
- just simply search the current version number in all files eg. 0.37.3
- don't forget to update a version number in the silverqx/TinyORM-HelloWorld find_package() call
- TinyORM:
9 files must be modified, README.md, docs/README.mdx, two vcpkg.json files, NOTES.txt,
hello-world.mdx, migrations.mdx, tinyorm.mdx plus bumped version.hpp file
- tom:
5 files must be modified, README.md, docs/README.mdx, NOTES.txt, plus bumped version.hpp file
- increase in the following files
include/orm/version.hpp
tests/TinyUtils/src/version.hpp
tom/include/tom/version.hpp
- and also informational version in badges
README.md
docs/README.md
- vcpkg ports version number
cmake/vcpkg/ports/tinyorm/vcpkg.json
cmake/vcpkg/ports/tinyorm-qt5/vcpkg.json
- verions in docs
hello-world.mdx#cmake-project
hello-world.mdx#fetchcontent
migrations.mdx#cmake-project
tinyorm.mdx#consume-tinyorm-library-cmake
How to create a new Gentoo tinyorm ebuild:
------------------------------------------
cd /var/db/repos/crystal/dev-db/tinyorm/
sudo cp/mv tinyorm-0.37.x.ebuild tinyorm-0.37.y.ebuild
sudo ebuild ./tinyorm-0.37.y.ebuild manifest
copy an updated Manifest and a new ebuild to tools/distributions/gentoo/repos/crystal/dev-db/tinyorm/
commit to TinyORM project
sudo emerge --update --newuse --deep --quiet-build -a @world
Number of Unit tests:
---------------------
- Qt5 has 14 unit tests less; the QSQLITE driver tests are excluded in the tst_migrate;
Not SKIPPED but excluded using preprocessor macros!
Testing of the Qt 5 QSQLITE driver is excluded because it doesn't support
ALTER TABLE DROP COLUMN, support for dropping columns was added
in the SQLite v3.35.0.
- Linux has 3 unit tests less and 3 SKIPPED unit tests more;
checking exe properties are excluded in the tst_versions;
They are SKIPPED!
Linux doesn't have exe properties like version, description, ...
How to updated vcpkg tinyorm and tinyorm-qt5 ports:
---------------------------------------------------
Everything needed is in the tools/deploy.ps1 script so I write only a small summary.
Important are the vcpkg_from_github() REF and SHA512 options in the portfile.cmake.
Prefer tags in the REF but can also be a commit ID.
The SHA512 is a hash of the source code tinyorm.tar.gz archive, the tools/Get-VcpkgHash.ps1
script can be used to obtain this hash. The URL to download this archive is:
https://github.com/silverqx/TinyORM/archive/v0.37.3.tar.gz
https://github.com/silverqx/TinyORM/archive/ca8909896247b21bf08d62a5109b23e9f65c89e1.tar.gz
If only the vcpkg is updated but the TinyORM version number is not bumped then
the port-version field must be added or bumped in the vcpkg.json file.
But all of this is handled by the tools/deploy.ps1 script. 👌
New library/executable added to the TinyORM project:
----------------------------------------------------
- define the private C macro TINYORM_PRAGMA_SYSTEM_HEADER_OFF for every new library/executable
added to the TinyORM project; if this is not the case, all warnings will be suppressed,
so this needs a special care
Proxy methods that internally call the toBase() (applySoftDeletes):
-------------------------------------------------------------------
I have little misunderstood when the SoftDeletes constraint will be applied, I thought that it will be applied only
during the TinyBuilder::toBase(), that is not absolutely true because whether it will be applied is controlled by
the Model::newQueryXyz() methods. These Model::newQueryXyz() methods are called on various places, somehwhere
the newQuery() (applies SoftDeletes) is called and on another places the newModelQuery() (doesn't apply SoftDeletes) or newQueryWithoutRelationships() is called.
Anyway I leave this new structure of proxy methods that are divided by the toBase() or getQuery() internal method calls,
it makes sense.
Proxy methods on the TinyBuilder that need to apply SoftDeletes (call toBase() internally).
aggregate
average
avg
count
dd
decrement
doesntExist
doesntExistOr
dump
exists
existsOr
explain (currently not implemented)
implode
increment
max
min
pluck
remove
sum
toSql
update
upsert
These three getters are discutable, but I will apply SoftDeletes, it won't hurt anything:
from
getConnection
getGrammar
There is not need to apply SoftDeletes for the getBindings() as the BuildsSoftDeletes is not adding any new bindings:
getBindings
There is not need to apply SoftDeletes on insert methods:
insert
insertGetId
insertOrIgnore
insertUsing
There is not need to apply SoftDeletes on these:
raw
QDateTime and database date/time types:
---------------------------------------
The TinyORM handles the QDateTime's time zone correctly, it converts a time zone to the time zone
defined in the "qt_timezone" connection configuration option and then it converts a QDateTime
instance to a string using the Model::u_dateFormat,
look at the Model::setAttribute() -> Model::fromDateTime().
Also, look at the commit that has an extensive description:
QDateTime overhaul 🤯🤐🙃 (1ded27bb)
MySQL:
---
Qt QMYSQL driver:
It's not so simple, first, it ignores the QDateTime timezone, it simply takes
a DateTime value that was given during the QDateTime creation (ctor/fromString/...),
and exactly this value will be sent to the MySQL database.
datetime column type:
- will save exactly the same value that was sent to the database
timestamp column type:
- is another story, here is important the MySQL timezone session variable,
the MySQL server converts the DateTime value that was sent to the UTC timezone
for storage and this UTC value will be saved in the database, the same is true during
retrieval of this value, so it converts it from the UTC to a session timezone
Database:
MySQL database converts TIMESTAMP values from the current timezone to UTC
for storage, and back from UTC to the current timezone for retrieval.
This does not occur for other types such as DATETIME.
By default, the current timezone for each connection is the server's time.
The timezone can be set on a per-connection basis using the time_zone system
variable.
Summary:
Sent:
It IGNORES the QDateTime timezone!
MYSQL_TIME *myTime = toMySqlDate(val.toDate(), val.toTime(), val.userType());
And the toMySqlDate() function picks internally the year, month, hour, minute, ...
one by one, so it calls date.year(), date.month(), time.hour(), ...
Retrieval:
Returns in the local timezone.
- for prepared statements:
return QDateTime(date, time);
- for non-prepared/normal statements:
return qDateTimeFromString(val); it internally calls
return QVariant(QDateTime::fromString(val, Qt::ISODate));
SQLite:
---
Qt QSQLITE driver:
It doesn't ignore the QDateTime timezone, it converts the QDateTime to a string using
.toString(Qt::ISODateWithMs) and this string value is sent to the database and
will be saved as a string column.
Database:
The SQLite database doesn't support DateTime-related types but has functions
for working with or converting DateTimes; they also allow to convert between
different timezones (using eg. strftime()).
Summary:
Sent:
It DOESN'T ignore the QDateTime timezone.
dateTime.toString(Qt::ISODateWithMs);
Retrieval:
It returns QVariant(QMetaType::QString).
- so it's up to a user how it will instantiate a QDateTime object from this
string value
PostgreSQL:
---
Qt QPSQL driver:
It's not so simple, it converts all QDateTime time zone to UTC and sends it to the database
in the ISO UTC format, for both for timestamps/datetime with or without time zone.
So the timestamps with time zone behave exactly like for the MySQL DB.
But timestamps without time zone don't, they are converted to the LOCAL DB server time zone
by DB before they are saved to DB storage.
Database:
The PostgreSQL server timezone can be set using the set TIME ZONE or set TIMEZONE TO
session variable or by the PGTZ environment variable, or of course using the main
configuration file and using the timezone configuration setting eg.
timezone = 'Europe/Bratislava'.
Summary:
Sent:
It DOESN'T ignore the QDateTime timezone.
// we force the value to be considered with a timezone information, and we force it to be UTC
// this is safe since postgresql stores only the UTC value and not the timezone offset (only used
// while parsing), so we have correct behavior in both case of with timezone and without tz
r = QStringLiteral("TIMESTAMP WITH TIME ZONE ") + QLatin1Char('\'') +
QLocale::c().toString(field.value().toDateTime().toUTC(), u"yyyy-MM-ddThh:mm:ss.zzz") +
QLatin1Char('Z') + QLatin1Char('\'');
Retrieval:
return QVariant(QDateTime::fromString(QString::fromLatin1(val),
Qt::ISODate).toLocalTime());
Handling NULL values:
---------------------
Also look qsqlresult.cpp -> QSqlResultPrivate::isVariantNull()
Also, look at the commit:
tests QDateTime null values (e8b3c3c5)
MySQL:
---
Summary:
Sent:
if (field.isNull())
r = QStringLiteral("NULL");
Retrieval:
return QVariant(f.type);
SQLite:
---
Summary:
Sent:
if (QSqlResultPrivate::isVariantNull(value))
res = sqlite3_bind_null(d->stmt, i + 1);
Retrieval:
case SQLITE_NULL:
values[i + idx] = QVariant(QMetaType::fromType<QString>());
PostgreSQL:
---
Summary:
Sent:
const auto nullStr = [](){ return QStringLiteral("NULL"); };
QString r;
if (field.isNull())
r = nullStr();
Retrieval:
if (PQgetisnull(d->result, currentRow, i))
return QVariant(type, nullptr);
Handling integer values:
------------------------
bool, smallint, bigint, ... are database types, _u suffix means unsigned.
Int, Short, LongLong, ... are QMetaType-s.
bool smallint smallint_u int int_u bigint bigint_u double decimal
MySQL Int Short UShort Int UInt LongLong ULongLong Double Double
PostgreSQL Bool Int - Int - - - Double Double
SQLite LongLong LongLong LongLong LongLong LongLong LongLong LongLong Double Double
Special rules:
-----
PostgreSQL:
---
- doesn't have unsigned integer numbers.
- returns Int for all types <Int
- bigint has special handling, it returns LongLong for positive numbers and ULongLong for negative,
it simply detects - character at the beginning
null values:
-----
In this case Int, Short, LongLong, ... are QMetaType-s that are passed to the QVariant ctor to
create a null QVariant eg. QVariant(QMetaType(QMetaType::Int)).
bool smallint smallint_u int int_u bigint bigint_u double decimal
MySQL Char Short UShort Int UInt LongLong ULongLong Double Double
PostgreSQL Bool Int - Int - LongLong LongLong Double Double
SQLite QString QString QString QString QString QString QString QString QString
Special rules:
-----
- QSQLITE driver - returns null QVariant(QMetaType(QMetaType::QString)) for all null values
- QMYSQL driver - returns null QVariant(QMetaType(QMetaType::Char)) for tinyint database types
- QPSQL driver - the logic for - and ULongLong is not applied for the null values, so it's
everytime LongLong
All places where is created some Model instance:
------------------------------------------------
The following list contains only a direct Model's constructor calls like "Derived model;",
these direct constructor calls have been replaced everywhere to Model::instance() related
methods to support the Default Attribute values.
All instance allocation are on the stack unless otherwise noted.
Model:
ctors
instance()
on()
query()
newTinyBuilder()
newFromBuilder()
newInstance()
HasAttributes:
getOriginal()
HasRelationships:
newRelatedInstance() - heap allocation
ModelProxies:
destroy()
BasePivot:
fromAttributes()
Orm::DatabaseConnection smart pointers data members graph:
----------------------------------------------------------
The order is also correct in the following graph, I'm visualizing it to better understand and try
to avoid shared pointers reference cycles.
- DatabaseConnection : public std::enable_shared_from_this<DatabaseConnection>
- shared_ptr<QueryGrammar> m_queryGrammar
- shared_ptr<SchemaGrammar> m_schemaGrammar
- unique_ptr<SchemaBuilder> m_schemaBuilder
- shared_ptr<DatabaseConnection> m_connection
- shared_ptr<SchemaGrammar> m_grammar
- unique_ptr<QueryProcessor> m_postProcessor
- QueryBuilder
- std::shared_ptr<DatabaseConnection> m_connection
- std::shared_ptr<QueryGrammar> m_grammar
- SchemaBuilder
- std::shared_ptr<DatabaseConnection> m_connection
- std::shared_ptr<QueryGrammar> m_grammar
- Relation
- std::shared_ptr<Related> m_related
- TinyBuilder
- std::shared_ptr<QueryBuilder> m_query
C preprocessor macros, defines, private, public, interface:
-----------------------------------------------------------
- qmake:
- TINYORM_MYSQL_PING and linking against MySQL C library:
- TINYORM_MYSQL_PING is public macro and should also be defined in the application that
consumes TinyORM library. If the TinyORM library was build with the mysql_ping config. then
it should also be defined in the consumer application. But, if the mysql_ping is not defined
Nothing Bad Happens because the TINYORM_MYSQL_PING is only used in the databaseconnection.hpp
to #ifdef the friend MySqlConnection declaration what means that it doesn't affect
a consumer application in any sense.
- linking against MySQL C library is not needed in the TinyOrm.pri and tom.pri
Todos to check in TinyOrm:
--------------------------
- QueryBuilder::insertGetId() allows insert with empty attributes, also Model::performInsert()
when incrementing == true, but all other insert methods don't, it's big inconsistency, unify it
Documentation TinyOrm Todos:
----------------------------
- how to refer NULL in docs, for now I leave it NULL
TODOs which look bad in code:
-----------------------------
- future add onDelete (and similar) callback feature
/*! Delete records from the database. */
void deleteModel()
{
// TODO future add onDelete (and similar) callback feature silverqx
// if (isset($this->onDelete)) {
// return call_user_func($this->onDelete, $this);
// }
return toBase().deleteRow();
}
- add c++20 compiler check, something like:
#ifdef __cplusplus
# if __cplusplus < 201103L && !defined(Q_CC_MSVC)
# error Qt requires a C++11 compiler and yours does not seem to be that.
# endif
#endif
- check this in cmake build:
#include(GenerateExportHeader)
#_test_compiler_hidden_visibility()
Todo categories:
----------------
Common:
- api different : different api than Laravel's Eloquent
- check : something to find out 🤔
- concept : add concept or constraint
- docs : document code or update markdown documentation
- desirable : feature which is extremely wanted
- dilemma : some sort of a fuckup
- duplicate : duplicate code
- feature : some feature to implement, perpend before feature described below
- future : task which has lower priority, because still much to do
- mistake : bad decision during prototyping 😭
- move : c++ move semantics
- mystery : don't know why that stuff is happening, find out what's up
- now : do it before commit
- next : next thing in the row to do after commit
- overflow : add check code, eg when size_t to int conversion
- perf : performance
- production : check before deploy to production
- regex : mark regex-es, try to avoid them
- reliability : make things more robust and reliable
- repeat : tasks that should I make from time to time
- security : self explaining
- study : don't know how something works, need to check up
- sync : synchronization in multi thread environment silverqx
- test : tasks in auto tests
- types : juggling with c++ types
Features related/to implement:
- aggregates : aggregate values like count, max, min, avg and sum
- castable : attributes casting
- default attributes : Default Attribute Values
- dilemma primarykey : different types for primary keys
- expression : DB::raw() support in the query builder
- events : event system
- ga : github actions
- guarded : related to the mass assignable feature
- json columns : JSON support
- logging : logging related
- migrations : database migrations related
- multidriver : task related to adding support for another drivers PostgreSQL, SQLite and SQL Server
- pivot : pivot table in the many-to-many relationship
- postgres : specific to PostgreSQL server
- qt6 : related to Qt6 upgrade or compatibility
- read/write connection : read/write connection
- relations : relations related 🤓
- savepoints : database savepoints
- scopes : query scopes
- seeders : seeders related
- table prefix : table prefix in the query grammar
- tom : tom migrations related
Upgrade the Qt to new version:
------------------------------
- run the QtCreator and do the PrintScreen of the Settings - Kits - Kits
- close the QtCreator
- run the Qt Maintenance Tool and add a new Qt version and also remove the old Qt version if want
- remove the old C:\Qt\x.y.z version folder (leftovers)
- open dotfiles in the vscode and file/replace all occurences of x.y.z and x_y_z to the new version; commit - upgraded to Qt vx.y.z
- add a new Qt version on the user PATH environment variable
- build the QMYSQL driver - qtbuild-qmysql-driver.ps1 6.7.0
- open the QtCreator
- fix all the Settings - Kits - Kits Qt versions
- fix the Settings - Debugger - Source Paths Mapping
- open the Projects mode (ctrl+5) for TinyORM and TinyOrmPlayground and disable/enable kit to update all build paths
- rebuild TinyORM and invoke unit tests
GitHub Action workflows:
- open TinyORM in the vscode and file/replace all occurrences of x.y.z and x_y_z to the new version
- commit - workflows upgraded to Qt vx.y.z
- TinyOrm-files:
- 7zip the qsqlmysql.dll, qsqlmysql.pdb, qsqlmysqld.dll, qsqlmysqld.pdb to the TinyOrm-files/qmysql_dlls-x.y.z.7z
- remove the old TinyOrm-files/qmysql_dlls-x.y.z.7z file
- generate the new hash - .\tools\Get-DownloadsHash.ps1 -Platform Linux, Windows
- commit - upgraded to Qt v6.7.0
- update the URL_QMYSQL_DLLS_MSVC_X64_x_y_z GitHub secret
MySQL Timezone tables:
- update the URL_MYSQL_TIMEZONE_TABLES GitHub secret
- download from https://dev.mysql.com/downloads/timezones.html
- pick the posix version without the leap second
- Commands to update MySQL server on merydeye-devel
mysql_config_editor print --all
mysql --login-path=root -p mysql
source timezone_posix.sql;
Restart-Service MySQL84
Versions info:
--------------
This is laravel/framework version, not laravel/laravel version:
- I have cloned repository at - E:\htdocs\laravel-src-master
- based on Laravel v8.26.1
- upgrade to Laravel v8.41.0 ( 15.5.2021, but I didn't merged/reflected new changes to TinyORM )
- upgrade to Laravel v8.80.0 ( 19.1.2021, upgrade from v8.41.0 )
- compare URLs (remove after merge):
- https://github.com/laravel/framework/compare/v8.26.1...v8.41.0
- https://github.com/laravel/framework/compare/v8.41.0...v8.80.0
- upgraded to Laravel 9
- v9.44.0 to v9.48.0
- v9.48.0 to v9.50.2
- upgraded to Laravel 10
- v9.50.2 to v10.0.3
- v10.9.0 to v10.14.1
- close all expanded sections in github compare diff:
document.querySelectorAll('.btn-octicon.js-details-target[aria-expanded=true]').forEach((value) => value.click())
Maintenance:
------------
- from time to time try:
- compile without PCH
- compile with Qt6, I have still problem with clazy
GitHub Actions:
---------------
- Repeatedly
- delete all _work\TinyORM\TinyORM-builds-cmake\Drivers-msys2-u-* after bumping versions because
tst_versions test case will fail, I think ccache doesn't get these bumped C macro versions and
then it fails
- invoke workflows manually (take spacial care on which branch is a workflow invoked):
gh workflow run --ref silverqx-develop
gh workflow run --ref main
- msvc2022-qt6-drivers.yml is invoked automatically on:push:
- linux-qt6-drivers.yml must be invoked manually after the msys2-ucrt64-drivers.yml finishes
- msvc2022-qt6-drivers.yml and linux-qt6-drivers.yml are first/main/entrance workflows that
invoke other workflows internally using the 'gh workflow run' command to run them
synchronously one by one so the CMake parallel argument can be as high as possible
- Mass deletion of GitHub workflow runs
- authenticate using the gh if auth expired
- gh auth login -h github.com
- dwr silverqx/TinyORM
- <tab> or <shift-tab> to un/select
- <enter> to delete selected runs
- ~/.dotfiles/bin/dwr
- upgrade to the latest dwr
- https://qmacro.org/blog/posts/2021/03/26/mass-deletion-of-github-actions-workflow-runs/
- https://raw.githubusercontent.com/qmacro/dotfiles/230c6df494f239e9d1762794943847816e1b7c32/scripts/dwr
- Runner images: https://github.com/actions/runner-images/tree/main/images
- My common folders structure:
- checkout to the current folder
- the github.workspace is the current workspace folder
- if using more repo checkouts then checkout using side-by-side method:
https://github.com/actions/checkout?tab=readme-ov-file#Checkout-multiple-repos-side-by-side
- the runner.workspace points to the parent folder
- I'm using it for build folders or things that are generated during workflows
- the env.TinyRunnerWorkPath points two folders up ../..
- it's directly in the _work/ or work/ folder
- I'm using it for more common things like data from GH extensions or databases data folders
- all actions with versions currently used by TinyORM:
actions/cache@v3
actions/checkout@v4
actions/github-script@v6
ilammy/msvc-dev-cmd@v1
jurplel/install-qt-action@v3
KyleMayes/install-llvm-action@v1
lukka/get-cmake@latest
msys2/setup-msys2@v2
mxschmitt/action-tmate@v3
seanmiddleditch/gha-setup-ninja@master
Not using anymore:
Chocobo1/setup-ccache-action@v1
- periodical upgrades:
- analyzers-qtX.yml
- add-apt-repository Clang 14
- clang-cl-qt6.yml
- Install LLVM and Clang 15.0.6 (KyleMayes/install-llvm-action@v1)
- Qt 6.7.0 install base components (jurplel/install-qt-action@v3)
- QMYSQL install driver dlls (Qt 6.7.0) - needed to rebuild the QSQL QMYSQL driver
- linux-qtX.yml
- add-apt-repository Clang 15
- msvc-2022.yml
- Qt 6.7.0 install base components (jurplel/install-qt-action@v3)
- QMYSQL install driver dlls (Qt 6.7.0) - needed to rebuild the QSQL QMYSQL driver
- how to update the clazy-standalone (analyzers.yml):
- update the QtCreator to latest version, copy libexec/qtcreator/clang/ to some empty folder
- leave only the clazy-standalone executable in the bin/ and lib/ folder, and compress it:
tar cjvf ../clazy-standalone.tar.bz2 .
- then update this file in the https://github.com/silverqx/files project
- no need to udpate the URL in the GitHub URL_CLAZY_STANDALONE_LINUX_X64 secret,
it's still the same:
https://github.com/silverqx/files/raw/main/clazy-standalone.tar.bz2
- prepend to the env. variables:
- name: Print env
run: |
echo "LD_LIBRARY_PATH: $LD_LIBRARY_PATH"
echo "LIBRARY_PATH: $LIBRARY_PATH"
echo "PATH: $PATH"
- name: TinyORM prepend to the system $PATH
working-directory: ..
run: |
echo "LD_LIBRARY_PATH=$PWD${LD_LIBRARY_PATH:+:}$LD_LIBRARY_PATH" >> $GITHUB_ENV
echo "LIBRARY_PATH=$GITHUB_WORKSPACE" >> $GITHUB_ENV
pwd >> $GITHUB_PATH
- name: Print env
working-directory: ..
run: |
pwd
echo "LD_LIBRARY_PATH: $LD_LIBRARY_PATH"
echo "LIBRARY_PATH: $LIBRARY_PATH"
echo "PATH: $PATH"
- run: exit 1
- vcpkg debug:
- name: TinyORM cmake configure (msvc-cmake-debug)
run: >-
echo $env:VCPKG_ROOT
echo $env:VCPKG_DEFAULT_TRIPLET
echo $env:VCPKG_MAX_CONCURRENCY
cmake
-S .
-B ../TinyORM-builds-cmake/build-msvc-cmake-debug
...
- ssh into runner:
- shortcuts:
ctrl-b ? - help (show commands)
ctrl-b c - new window
ctrl-b l - last window (switch to)
ctrl-b n - next window (switch to)
ctrl-b p - previous window (switch to)
ctrl-b 0-9 - switch to window
ctrl-b [ - enable scroll mode (eg. Up Arrow or PgDn), press q to quit the scroll mode
ctrl-b " - split pane (horizontally)
ctrl-b % - split pane (vertically)
ctrl-b z - resize pane (toggle maximize)
ctrl-b M-up - resize pane (up, down, left, right; M- is L-ALT)
ctrl-b up - select pane (up, down, left, right)
- run if failure or success:
- name: Setup tmate session
if: ${{ always() }}
uses: mxschmitt/action-tmate@v3
with:
limit-access-to-actor: true
- run: exit 1
- !!! with if: ${{ failure() }} condition !!!
--------------------
- name: Setup tmate session
if: ${{ failure() }}
uses: mxschmitt/action-tmate@v3
with:
limit-access-to-actor: true
- run: exit 1
- all TinyORM env. variables
env:
DB_MYSQL_CHARSET: ${{ secrets.DB_MYSQL_CHARSET }}
DB_MYSQL_COLLATION: ${{ secrets.DB_MYSQL_COLLATION }}
DB_MYSQL_DATABASE: ${{ secrets.DB_MYSQL_DATABASE }}
DB_MYSQL_HOST: ${{ secrets.DB_MYSQL_HOST_SSL }}
DB_MYSQL_PASSWORD: ${{ secrets.DB_MYSQL_PASSWORD }}
DB_MYSQL_SSL_CA: ${{ runner.workspace }}/../mysql/data/ca.pem
DB_MYSQL_SSL_CERT: ${{ runner.workspace }}/../mysql/data/client-cert.pem
DB_MYSQL_SSL_KEY: ${{ runner.workspace }}/../mysql/data/client-key.pem
DB_MYSQL_SSL_MODE: ${{ secrets.DB_MYSQL_SSL_MODE }}
DB_MYSQL_USERNAME: ${{ secrets.DB_MYSQL_USERNAME }}
DB_PGSQL_CHARSET: utf8
DB_PGSQL_DATABASE: ${{ secrets.DB_PGSQL_DATABASE }}
DB_PGSQL_HOST: ${{ secrets.DB_PGSQL_HOST }}
DB_PGSQL_PASSWORD: ${{ secrets.DB_PGSQL_PASSWORD }}
DB_PGSQL_USERNAME: ${{ secrets.DB_PGSQL_USERNAME }}
DB_SQLITE_DATABASE: ${{ runner.temp }}/${{ secrets.DB_SQLITE_DATABASE }}
TOM_TESTDATA_ENV: testing
- print all contexts:
Linux:
- name: Dump pwd
run: pwd
- name: Dump OS env (unsorted)
run: env
- name: Dump OS env (sorted)
run: |
env | sort --ignore-case
Windows:
- name: Dump pwd
run: Get-Location
- name: Dump OS env
run: |
Get-ChildItem env:
Common:
- name: Dump GitHub context
env:
GITHUB_CONTEXT: ${{ toJSON(github) }}
run: echo "$GITHUB_CONTEXT"
- name: Dump job context
env:
JOB_CONTEXT: ${{ toJSON(job) }}
run: echo "$JOB_CONTEXT"
- name: Dump steps context
env:
STEPS_CONTEXT: ${{ toJSON(steps) }}
run: echo "$STEPS_CONTEXT"
- name: Dump runner context
env:
RUNNER_CONTEXT: ${{ toJSON(runner) }}
run: echo "$RUNNER_CONTEXT"
- name: Dump strategy context
env:
STRATEGY_CONTEXT: ${{ toJSON(strategy) }}
run: echo "$STRATEGY_CONTEXT"
- name: Dump matrix context
env:
MATRIX_CONTEXT: ${{ toJSON(matrix) }}
run: echo "$MATRIX_CONTEXT"
- name: Dump env context
env:
ENV_CONTEXT: ${{ toJSON(env) }}
run: echo "$ENV_CONTEXT"
- run: exit 1
- query caches using the gh api
# Print all caches usage summary
gh api -H "Accept: application/vnd.github+json" /repos/silverqx/TinyORM/actions/cache/usage
# List caches
gh api -H "Accept: application/vnd.github+json" /repos/silverqx/TinyORM/actions/caches
# Delete all caches for TinyORM repo
gh api --method DELETE -H "Accept: application/vnd.github+json" /repos/silverqx/TinyORM/actions/caches
# Delete a cache by ID for TinyORM repo
gh api --method DELETE -H "Accept: application/vnd.github+json" /repos/silverqx/TinyORM/actions/caches/CACHE_ID
- ternary operator alternative:
${{ matrix.compiler.key == 'gcc' && 1 || 2 }}
- use quotes for strings:
${{ matrix.compiler.key == 'gcc' && '400M' || '250M' }}
- ccache initial build sizes:
First value is with a default compression and second uncompressed (ccache --show-compression)
with PCH:
linux-gcc-qt64 - 780MB (5.0GB)
linux-clang-qt63 - 440MB (1.3GB)
linux-gcc-qt5 - 770MB (4.5GB)
w/o PCH:
clang-cl-qt64 - 135MB (880MB)
msvc2022-qt64 - 190MB (1.5GB)
msvc2019-qt5 - 175MB (1.5GB)
linux-gcc-qt64 - 165MB (700MB)
linux-clang-qt63 - 100MB (680MB)
linux-gcc-qt5 - 115MB (665MB)
linux-clang-qt5 - 90MB (640MB)
MSYS2-clang-qt5 - 140MB (900MB)
MSYS2-clang-qt6 - 140MB (895MB)
MSYS2-gcc-qt5 - 180MB (940MB)
MSYS2-gcc-qt6 - 180MB (940MB)
- ccache debugging (Linux):
- name: Ccache enable debugging
id: ccache-debug
run: |
ccacheDebugDir0='${{ runner.temp }}/ccache_debug/run_0'
ccacheDebugDir1='${{ runner.temp }}/ccache_debug/run_1'
mkdir -p "$ccacheDebugDir0"
mkdir -p "$ccacheDebugDir1"
echo "CCACHE_DEBUG=1" >> $GITHUB_ENV
echo "TinyCcacheDebugDir0=$ccacheDebugDir0" >> $GITHUB_OUTPUT
echo "TinyCcacheDebugDir1=$ccacheDebugDir1" >> $GITHUB_OUTPUT
... do build here
- name: TinyORM cmake build ✨ (${{ matrix.compiler.key }}-cmake-debug) (ccache debug 0)
run: >-
export CCACHE_DEBUGDIR='${{ steps.ccache-debug.outputs.TinyCcacheDebugDir0 }}'
cmake --build ../TinyORM-builds-cmake/build-${{ matrix.compiler.key }}-cmake-debug
--target all --parallel 2
- name: Ccache statistics
run: |
ccache --show-stats -vv
ccache --zero-stats
- name: Ccache upload debugging logs (ccache debug 0)
uses: actions/upload-artifact@v3
with:
name: ccache_debug_run_0
path: ${{ steps.ccache-debug.outputs.TinyCcacheDebugDir0 }}
if-no-files-found: error
- name: TinyORM cmake build ✨ (${{ matrix.compiler.key }}-cmake-debug) (ccache debug 1)
run: >-
cmake --build ../TinyORM-builds-cmake/build-${{ matrix.compiler.key }}-cmake-debug
--target clean
export CCACHE_DEBUGDIR='${{ steps.ccache-debug.outputs.TinyCcacheDebugDir1 }}'
cmake --build ../TinyORM-builds-cmake/build-${{ matrix.compiler.key }}-cmake-debug
--target all --parallel 2
- name: Ccache statistics (ccache debug 1)
run: |
ccache --show-stats -vv
ccache --zero-stats
- name: Ccache upload debugging logs (ccache debug 1)
uses: actions/upload-artifact@v3
with:
name: ccache_debug_run_1
path: ${{ steps.ccache-debug.outputs.TinyCcacheDebugDir1 }}
if-no-files-found: error
RegEx-s:
-------
- const data members:
(?<![\(\)])(const) +.* +\bm_.*\b( +.*)?;$
(?<![\(\)])(const) +.* +\bm_.*\b +=
(?<![\(\)])(const) +.* +\bm_.*\b +{
- const data member references:
(?<![\(\)])(const) +.* +&\bm_.*\b( +.*)?;$
- all exceptions:
throw (.*::)?\w+(E|_error)
Powershell commands:
--------------------
- export todos to csv:
Get-ChildItem -Path *.cpp,*.hpp -Recurse | sls -Pattern ' (TODO|NOTE|FIXME|BUG|WARNING|CUR|FEATURE|TEST|FUTURE) ' -CaseSensitive | % { $_.Line = $_.Line.Trim().TrimStart('// '); return $_; } | select Line,LineNumber,Path | Export-Csv todos.csv -Delimiter ';' -NoTypeInformation
- search in todos:
Get-ChildItem -Path *.cpp,*.hpp -Recurse | sls -Pattern ' (TODO|NOTE|FIXME|BUG|WARNING|CUR|FEATURE|TEST|FUTURE) ' -CaseSensitive | % { $_.Line = $_.Line.Trim().TrimStart('// '); return $_; } | where Line -Match 'pch' | select Line,LineNumber,Path | ft -AutoSize
- filter out executed queries:
Get-Content .\tmp.sql | sls -Pattern '^(Executed prepared query)' | Set-Content executed_queries.sql
- TinyOrmPlayground - run InvokeXTimes.ps1 on Linux:
stp && sq5
export LD_LIBRARY_PATH=../../../TinyORM/TinyORM-builds-qmake/build-TinyORM-Desktop_Qt_5_15_2_GCC_64bit_ccache-Debug/src
pwsh -NoLogo -NoProfile -File InvokeXTimes.ps1 2 ../../../TinyOrmPlayground/TinyOrmPlayground-builds-qmake/build-TinyOrmPlayground-Desktop_Qt_5_15_2_GCC_64bit_ccache-Debug/TinyOrmPlayground
- TinyORM - run InvokeXTimes.ps1 on Linux:
stp && sq5 && cdtq && cd build-TinyORM-Desktop_Qt_5_15_2_GCC_64bit_ccache-Debug
export LD_LIBRARY_PATH=./src:./tests/TinyUtils${LD_LIBRARY_PATH:+:}$LD_LIBRARY_PATH
export PATH=$HOME/Code/c/TinyORM/tools:$PATH
InvokeXTimes.ps1
pwsh -NoLogo -NoProfile -File InvokeXTimes.ps1 100
Powershell Clang analyzers:
---------------------------
qa-lint-tinyorm-qt5.ps1 is tailor-made for TinyORM project.
qa-clang-tidy.ps1, qa-clazy-standalone.ps1, qa-clazy-standalone-st.ps1 are more general, they can be used with any project, "-st" script calls raw clazy-standalone.exe.
run-clang-tidy.ps1, run-clazy-standalone.ps1 are raw Powershell wrappers around python run-clang-tidy/run-clazy-standalone.py python scripts.
Invoke the run-clang-tidy.ps1 manually:
--
run-clang-tidy.ps1 -use-color -extra-arg-before='-Qunused-arguments' -j=10 -p='E:\c\qMedia\TinyORM\TinyORM-builds-cmake\build-lint-qt6_Debug' -checks='-*,readability-convert-member-functions-to-static' '(?:src|tests)[\\\/]+.+?[\\\/]+(?!mocs_)[\w_\-\+]+\.cpp$'
- test *.hpp count
gci -Force -Recurse -Include *.hpp `
-Path E:\c\qMedia\TinyORM\TinyORM\include,E:\c\qMedia\TinyORM\TinyORM\examples,e:\c\qMedia\TinyORM\TinyORM\tom,E:\c\qMedia\TinyORM\TinyORM\tests | `
select -ExpandProperty FullName > b.txt
- test *.cpp count
cat .\compile_commands.json | sls '"file":' > a.txt
And use regex-es from analyzers-qt5/6.yml, eg.:
Get-Content .\a.txt | sls -Pattern '[\\\/]+(?:drivers|examples|orm|src|tests|tom)[\\\/]+.+?[\\\/]+(?!mocs_)[\w\d_\-\+]+\.cpp' | measure
QtCreator clangd/analyzers confusions:
--------------------------------------
To finally solve this confusions I tested it all.
- always use the latest clangd, cland-tidy, clazy (scoop, Gentoo pacakge)
- for both clangd and analyzers in QtCreator settings set Clang warnings to:
-Wall -Wextra -Wpedantic
- no need to use -Weffc++ as is really outdated and produces false positives, also
it does nothing with clangd, in clang docs is written this:
Synonym for -Wnon-virtual-dtor.