-
Notifications
You must be signed in to change notification settings - Fork 0
/
atom.xml
595 lines (322 loc) · 275 KB
/
atom.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>小菜的 Blog</title>
<link href="/atom.xml" rel="self"/>
<link href="https://riverye.com/"/>
<updated>2023-08-15T14:01:48.561Z</updated>
<id>https://riverye.com/</id>
<author>
<name>River-Ye</name>
</author>
<generator uri="http://hexo.io/">Hexo</generator>
<entry>
<title>如何在 Ubuntu Server 用 mailcow 建置郵件伺服器</title>
<link href="https://riverye.com/2023/08/15/How-to-Build-a-Mail-Server-with-Mailcow-on-Ubuntu-Server/"/>
<id>https://riverye.com/2023/08/15/How-to-Build-a-Mail-Server-with-Mailcow-on-Ubuntu-Server/</id>
<published>2023-08-14T16:00:00.000Z</published>
<updated>2023-08-15T14:01:48.561Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>這篇是紀錄如何自行架設郵件伺服器 (Mail Server) 的過程，使用開源的 mailcow 建置</p><h2><span id="說明">說明</span></h2><p>當初在找開源的 Email 軟體時，主要是比較 <a href="https://mailcow.email/" target="_blank" rel="noopener">mailcow</a> 和 <a href="https://roundcube.net/" target="_blank" rel="noopener">Roundcube</a> 這兩套，<br>需求為:</p><ol><li>收發信</li><li>有 WebMail 介面</li><li>方便管理所有 Email 的介面</li><li>權限管理功能</li><li>有 API 可以打</li><li>容易建置</li><li>容易升級</li><li>開源專案有持續地維護</li></ol><p>因此選擇以 mailcow 作為首選</p><p>若想要比較不同 Email 的服務差異，可以參考 <a href="https://www.saashub.com/compare-mailcow-vs-roundcube" target="_blank" rel="noopener">SaaShub</a> 或 <a href="https://www.libhunt.com/compare-mailcow-dockerized-vs-roundcubemail" target="_blank" rel="noopener">LibHunt</a> ，可以比較任兩套的軟體</p><p><img src="saashub_compare.png" alt></p><h2><span id="安裝步驟">安裝步驟</span></h2><p>以下建置過程是在 Ubuntu Server 22.04.2 LTS ，<a href="https://riverye.com/2023/07/29/Ubuntu-Server-Installation-Tutorial/">可參考這篇如何建置</a></p><h3><span id="1-設定-dns">1. 設定 DNS</span></h3><p>建議先設定好 DNS ，生效會需要一些時間<br>假設我的固定 IP 是 <code>123.45.67.89</code><br>假設我的 domain 是 <a href="https://riverye.com">riverye.com</a></p><table><thead><tr><th>Name</th><th>Type</th><th>Value</th></tr></thead><tbody><tr><td>mail</td><td>IN A</td><td>123.45.67.89</td></tr><tr><td>autoconfig</td><td>IN CNAME</td><td>smtp.riverye.com</td></tr><tr><td>autodiscover</td><td>IN CNAME</td><td>smtp.riverye.com</td></tr><tr><td>riverye.com</td><td>IN MX 10</td><td>smtp.riverye.com</td></tr></tbody></table><h3><span id="2-安裝-docker">2. 安裝 Docker</span></h3><p>照著 <a href="https://docs.docker.com/engine/install/ubuntu/" target="_blank" rel="noopener">Docker 官方文件</a>，複製貼上，執行即可<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> sudo apt-get update</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> sudo apt-get install ca-certificates curl gnupg</span></span><br><span class="line"></span><br><span class="line"><span class="meta">$</span><span class="bash"> sudo install -m 0755 -d /etc/apt/keyrings</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> sudo chmod a+r /etc/apt/keyrings/docker.gpg</span></span><br><span class="line"></span><br><span class="line"><span class="meta">$</span><span class="bash"> <span class="built_in">echo</span> \</span></span><br><span class="line"> "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \</span><br><span class="line"> "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \</span><br><span class="line"> sudo tee /etc/apt/sources.list.d/docker.list > /dev/null</span><br><span class="line"></span><br><span class="line"><span class="meta">$</span><span class="bash"> sudo apt-get update</span></span><br><span class="line"></span><br><span class="line"><span class="meta">$</span><span class="bash"> sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin -y</span></span><br></pre></td></tr></table></figure></p><h3><span id="3-下載-mailcow">3. 下載 mailcow</span></h3><p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> sudo su -</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> <span class="built_in">cd</span> /opt</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> git <span class="built_in">clone</span> https://github.com/mailcow/mailcow-dockerized</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> <span class="built_in">cd</span> mailcow-dockerized</span></span><br></pre></td></tr></table></figure></p><p><img src="mailcow_01_%E4%B8%8B%E8%BC%89mailcow.png" alt></p><h3><span id="4-設定-mailcow-的-generate_configsh-和-mailcowconf">4. 設定 mailcow 的 <code>generate_config.sh</code> 和 <code>mailcow.conf</code></span></h3><h4><span id="4-1-設定-generate_configsh">4-1. 設定 <code>generate_config.sh</code></span></h4><p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> ./generate_config.sh</span></span><br><span class="line"></span><br><span class="line">Found Docker Compose Plugin (native).</span><br><span class="line">Setting the DOCKER_COMPOSE_VERSION Variable to native</span><br><span class="line">Notice: You´ll have to update this Compose Version via your Package Manager manually!</span><br><span class="line">Press enter to confirm the detected value '[value]' where applicable or enter a custom value.</span><br><span class="line">Mail server hostname (FQDN) - this is not your mail domain, but your mail servers hostname: mail.riverye.com</span><br><span class="line">Timezone [Asia/Taipei]:</span><br><span class="line">Which branch of mailcow do you want to use?</span><br><span class="line"></span><br><span class="line">Available Branches:</span><br><span class="line">- master branch (stable updates) | default, recommended [1]</span><br><span class="line">- nightly branch (unstable updates, testing) | not-production ready [2]</span><br><span class="line">Choose the Branch with it´s number [1/2] 1</span><br><span class="line">Fetching origin</span><br><span class="line">Your branch is up to date with 'origin/master'.</span><br><span class="line">Generating snake-oil certificate...</span><br><span class="line">..+.+...+.....+.......+............+...........+.+......+.....+...............+....+........+.+.....+..........+......+...+...+........+...........................+...+.+.....+....+..+...+....+..+.+.....+...............+.......+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.........+..+...+.+.................+...+....+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*....+.+...+.........+...+.........+...+..+.........................+.....+....+.....+.+.....+......+...+....+...+..+...+...+....+..+................+.....+............+.+.........+............+.....+..................+.+......+......+.........+.....+....+..+.......+..+..........+.....+....+..+....+......+...+......+.....+.+..+......+.......+.....+....+.....+.+.........+...+........+...............+.......+..................+.....+...+.+.....+..........+......+...+.........+.....+.......+...+......+...........+...+...+.......+...........+.+.........+..............................+.........+............+...+.......................+.........+...+......+.........+.+..+.........+.+..+......+...............+....+...+.............................+.+.................+.+....................+...............+.+......+.....+............+............................+............+.....+..........+.....+...+...+.........+............+..........+.....+.........+.......+...+.....+............+.........+...+...............+......+.........+............+......+...+.+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++</span><br><span class="line">...+.....+...+.............+.....+.......+...+..+....+.....+.........+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...................+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*..+....+...........+......+...+.+...+...........................+......+.........+...+.................+....+.....................+..............+.......+.....+....+.....+.+..................+..+...+......+...+......+......+.............+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++</span><br><span class="line">-----</span><br><span class="line">Copying snake-oil certificate...</span><br><span class="line">Detecting if your IP is listed on Spamhaus Bad ASN List...</span><br><span class="line">Check completed! Your IP is clean</span><br></pre></td></tr></table></figure></p><p><img src="mailcow_02_%E8%A8%AD%E5%AE%9Agenerate_config.png" alt></p><h4><span id="4-2-設定-mailcowconf">4-2. 設定 <code>mailcow.conf</code></span></h4><p>以下是我有調整的部分，可依照需求自行更改<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> vim mailcow.conf</span></span><br><span class="line"></span><br><span class="line">DBNAME=mailcow_db</span><br><span class="line">DBUSER=mailcow_user</span><br><span class="line"></span><br><span class="line">SKIP_IP_CHECK=y</span><br><span class="line">SKIP_HTTP_VERIFICATION=y</span><br><span class="line">ALLOW_ADMIN_EMAIL_LOGIN=y</span><br></pre></td></tr></table></figure></p><h3><span id="5-啟動服務">5. 啟動服務</span></h3><p>上述設定完成後，接著要用 Docker 啟動服務<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> docker compose pull</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> docker compose up -d</span></span><br></pre></td></tr></table></figure></p><h4><span id="5-1-服務啟動失敗">5-1. 服務啟動失敗</span></h4><p>啟動服務後，若開啟網頁看到以下錯誤，請不要慌，靜待數秒至幾分鐘後 (依電腦效能)，再重新整理，就能看到登入畫面了<br><img src="mailcow_03_%E9%96%8B%E5%95%9F%E5%A4%B1%E6%95%97.png" alt></p><h4><span id="5-1-服務啟動成功">5-1. 服務啟動成功</span></h4><p>看到以下畫面，代表服務有正常被啟動成功，若仍只看到上述畫面，可以照上面的錯誤訊息，印出 Log 看錯誤訊息</p><table><thead><tr><th style="text-align:center">預設帳號</th><th style="text-align:center">預設密碼</th></tr></thead><tbody><tr><td style="text-align:center">admin</td><td style="text-align:center">moohoo</td></tr></tbody></table><p>登入後，建議把預設密碼改掉<br><img src="mailcow_04_%E9%96%8B%E5%95%9F%E6%88%90%E5%8A%9F.png" alt></p><h3><span id="6-新增域名">6. 新增域名</span></h3><p>新增我的 domain<br>操作步驟：E-Mail -> Configuration -> 新增域名</p><p>以下是範例<br><img src="mailcow_05_%E6%96%B0%E5%A2%9E%E5%9F%9F%E5%90%8D.png" alt></p><h3><span id="7-dns-設定">7. DNS 設定</span></h3><p>操作步驟：E-Mail -> Configuration -> 找到剛才新增的域名(domain) -> DNS<br>依照提示去設定 DNS 即可</p><p>附上從 Cloudflare 匯出的 DNS 設定</p><p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line">;; A Records</span><br><span class="line">mail.riverye.com.1INA123.45.67.89</span><br><span class="line"></span><br><span class="line">;; CNAME Records</span><br><span class="line">autoconfig.riverye.com.1INCNAMEmail.riverye.com.</span><br><span class="line">autodiscover.riverye.com.1INCNAMEmail.riverye.com.</span><br><span class="line"></span><br><span class="line">;; MX Records</span><br><span class="line">riverye.com.1INMX10 mail.riverye.com.</span><br><span class="line"></span><br><span class="line">;; SRV Records</span><br><span class="line">_autodiscover._tcp.riverye.com.1INSRV10 10 443 mail.riverye.com.</span><br><span class="line"></span><br><span class="line">;; TLSA Records</span><br><span class="line">_25._tcp.mail.riverye.com.1INTLSA3 1 1 af728af706fe3eb0c23ad4387d2d24ad81cfa5811e3821f8fa206452ea52ef32</span><br><span class="line"></span><br><span class="line">;; TXT Records</span><br><span class="line">dkim._domainkey.riverye.com.1INTXT"v=DKIM1;k=rsa;t=s;s=email;p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1NRc1H0Ae7+vv//hwsXCKwGDDs6QnU8ZNZ/RbBCPATEMdNkfxrAhlY9Q9x4znQFdoXm6i6P2P39/EF/+PjaKMNR7YKwxe/LDfuZUMitR2/vccRxmRttoJ/FoEO2HimlTliGYE8ertDXHsmyS2Lf3ddxUbS4PZyHm18heWcoSA4IXmsL02JK0Jnt5aotqMAMyZk8m8WGkQkMfHyc3KdhcqfsCdfMBhRMp6HY72qrjRvjKWnQPeyZaB9mqeaasxjCTfZHOoEfi+2qJf04U3SYBOuhs1lv/BJXNJo5iyTHz9mBdPDqGFsn653BHcSpggEW6RXqctIOSMSlypW5/hcp8fQIDAQAB"</span><br><span class="line">_dmarc.riverye.com.1INTXT"v=DMARC1; p=none;"</span><br><span class="line">riverye.com.1INTXT"v=spf1 mx ~all"</span><br></pre></td></tr></table></figure></p><table><thead><tr><th>Name</th><th>Type</th><th>Value</th></tr></thead><tbody><tr><td>mail.riverye.com</td><td>IN A</td><td>123.45.67.89</td></tr><tr><td>_25._tcp.mail.riverye.com</td><td>IN TLSA 3 1 1</td><td>af728af706fe3eb0c23ad4387...</td></tr><tr><td>riverye.com</td><td>IN MX 10</td><td>mail.riverye.com</td></tr><tr><td>autodiscover.riverye.com</td><td>IN CNAME</td><td>mail.riverye.com</td></tr><tr><td>_autodiscover._tcp.riverye.com</td><td>INSRV10 10 443</td><td>mail.riverye.com</td></tr><tr><td>autoconfig.riverye.com</td><td>IN CNAME</td><td>mail.riverye.com</td></tr><tr><td>riverye.com</td><td>IN TXT</td><td>v=spf1 mx ~all</td></tr><tr><td>_dmarc.riverye.com</td><td>IN TXT</td><td>v=DMARC1; p=none;</td></tr><tr><td>dkim._domainkey.riverye.com</td><td>IN TXT</td><td>v=DKIM1;k=rsa;t=s;s=email;p=MIIBIjA...</td></tr></tbody></table><p><img src="mailcow_06_DNS%E8%A8%AD%E5%AE%9A.png" alt></p><p>設定完成後，等待 DNS 生效後 (快則數秒，慢則數小時)，可透過 mailcow 的介面檢查設定是否正確，正確則會有綠色 v<br><img src="mailcow_07_%E6%9F%A5%E7%9C%8BDNS%E8%A8%AD%E5%AE%9A%E7%B5%90%E6%9E%9C.png" alt></p><p>若已經設定好 DNS ，但在 mailcow 介面看仍尚未正確，可以在終端機使用 <code>dig</code> 指令 double check 看是否有輸入錯誤<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash"> 想看 SRV Type</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> dig SRV _autodiscover._tcp.riverye.com</span></span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 用特定的 DNS 服務查詢，以 Cloudflare 為例</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> dig @1.1.1.1 SRV _autodiscover._tcp.riverye.com</span></span><br></pre></td></tr></table></figure></p><h3><span id="8-新增信箱及測試寄信">8. 新增信箱及測試寄信</span></h3><p>影片教學<br><div class="video-container"><iframe src="//www.youtube.com/embed/KQXjtFTUpq4" frameborder="0" allowfullscreen></iframe></div></p><p>已經新增專屬的信箱後，再來就是要寄到外部，確認能否寄信成功<br>推薦可以使用 <a href="https://www.mail-tester.com/" target="_blank" rel="noopener">Newsletters spam test by mail-tester.com</a> 進行測試</p><p>以下是我的寄信評分<br><img src="mailcow_08_%E5%AF%84%E4%BF%A1%E8%A9%95%E5%88%86.png" alt></p><h2><span id="如何綁定-outlook">如何綁定 Outlook</span></h2><p>以 IMAP 為例，設定如下</p><table><thead><tr><th style="text-align:center">名稱</th><th style="text-align:center">伺服器</th><th style="text-align:center">連結埠</th><th style="text-align:center">加密方式</th></tr></thead><tbody><tr><td style="text-align:center">內送郵件</td><td style="text-align:center">mail.riverye.com</td><td style="text-align:center">993</td><td style="text-align:center">SSL/TLS</td></tr><tr><td style="text-align:center">外送郵件</td><td style="text-align:center">mail.riverye.com</td><td style="text-align:center">465</td><td style="text-align:center">SSL/TLS</td></tr></tbody></table><p><img src="Outlook_IMAP.png" alt></p><h2><span id="qa">QA.</span></h2><h3><span id="qa1-使用hinet非固定制固定ip並非固定制也可以自行架設嗎">QA1. 使用「HiNet非固定制固定IP」，並非「固定制」也可以自行架設嗎?</span></h3><p>可以的，只差別在 DNS 的 PTR 類型無法設定，常用的信箱 (ex: Gmail、Hotmail、Apple icloud 等) 還是能收到信，但如果是檢查比較嚴格的信箱，則可能會被拒收 (reject)，可依照需求評估是否要改用「固定制」。</p><p>在建置時，有遇到同樣的設定，使用固定 IP A 可以寄到上述所說的信箱，但換成另一組固定 IP 時，則全都被拒收，好在退信的信件中，有提供申訴的管道，提出申訴後，約等 3-4 小時，就能正常寄給上面的那些外部信箱了。</p><h3><span id="qa2-如何讓輸入的網址連到對應的電腦">QA2. 如何讓輸入的網址，連到對應的電腦</span></h3><p>關鍵字是「Port Forwarding (通訊埠轉發)」，也有廠商是用「虛擬伺服器」、「虛擬 IP」...等稱呼，指的是同樣的東西。</p><p>在 ASUS 的教學文件，寫的很清楚什麼是 Port Forwarding ，以及要如何設定 ASUS 的路由器<br><a href="https://www.asus.com/tw/support/FAQ/114093/" target="_blank" rel="noopener">[無線路由器]如何在華碩路由器設定虛擬伺服器(Port forwarding)？</a></p><h4><span id="asus-的設定範例">ASUS 的設定範例</span></h4><p>在 ASUS 中，是在「虛擬伺服器」中設定<br><img src="asus_port_forwarding.png" alt></p><h4><span id="fortinet-fortigate-的設定範例">Fortinet FortiGate 的設定範例</span></h4><p>在 FortiGate 中，是在「虛擬 IP」中設定</p><p><img src="fortigate_port_forwarding.png" alt></p><p>防火牆也要記得設定，免得無法將資料正確傳到指定的電腦，其中 NAT 要停用!!!<br>不然 Server 都只會接收到 Gateway IP Address ，無法知道真實的 IP，進而無法用 Fail2ban<br><img src="fortigate_firewall_01.png" alt></p><p><img src="fortigate_firewall_02.png" alt></p><h3><span id="qa3-路由器已經將-80-或-443-port-埠-用做其他用途該怎麼辦">QA3. 路由器已經將 80 或 443 port (埠) 用做其他用途，該怎麼辦？</span></h3><p>關鍵字是「Reverse Proxy (反向代理)」，可以使用 Nginx、Apache 來架設 Reverse Proxy<br>這邊推薦使用 <a href="https://caddyserver.com/docs/quick-starts/reverse-proxy" target="_blank" rel="noopener">Caddy</a>，比前 2 個更好設定、語法更簡潔、不用處理憑證 (CA)</p><p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash"> /etc/caddy/Caddyfile</span></span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 假設 Reverse Proxy Server 和 Mail Server 是不同電腦</span></span><br><span class="line">mail.riverye.com {</span><br><span class="line"> reverse_proxy 192.168.1.30</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 假設 Reverse Proxy Server 和 Mail Server 是同一台電腦，且要指定 port 給 Mail Server 使用</span></span><br><span class="line">mail.riverye.com {</span><br><span class="line">reverse_proxy localhost:9527</span><br><span class="line">}</span><br></pre></td></tr></table></figure></p><h3><span id="qa4-如何看某-port-的流量紀錄">QA4. 如何看某 port 的流量紀錄?</span></h3><p>會特別提到這個，是因為當初沒注意到 Fortinet FortiGate 的 NAT 有啟用，因此用 API 打 mailcow 時，雖然有設定 IP 檢查，卻沒照預期運作。<br>才會有需要看某 port 的流量紀錄需求，可在終端機中使用 <code>tcpdump</code> 指令。</p><p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash"> 想看 443 port</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> tcpdump -f port 443</span></span><br></pre></td></tr></table></figure></p><h3><span id="qa5-如何將-mailcow-更新到最新">QA5. 如何將 mailcow 更新到最新?</span></h3><p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> <span class="built_in">cd</span> /opt/mailcow-dockerized</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> ./update.sh</span></span><br></pre></td></tr></table></figure></p><h3><span id="qa6-如何避免有人惡意想要嘗試登入後台">QA6. 如何避免有人惡意想要嘗試登入後台?</span></h3><p>可以使用 Fail2ban， mailcow 的介面，能直接設定</p><p><img src="mailcow_09_fail2ban.png" alt></p><hr><h2><span id="小結">小結</span></h2><p>自行架設的過程當中，會學到許多知識，尤其是硬體到軟體都自己弄時，會發現自己不會的東西很多，而不停地去瞭解接觸到的關鍵字，趁記憶猶新時，寫下此篇紀錄，以防之後忘了</p><hr><h2><span id="參考文件">參考文件</span></h2><ol><li><a href="https://docs.mailcow.email/" target="_blank" rel="noopener">mailcow: dockerized documentation</a></li><li><a href="https://youtu.be/czSJqYor-xM" target="_blank" rel="noopener">Install MailCow Server on Ubuntu 22 04 x64</a></li><li><a href="https://u.sb/docker-mailcow/" target="_blank" rel="noopener">Debian 11 / Ubuntu 22.04 使用 Docker 安装 Mailcow 自建域名邮箱</a></li><li><a href="https://blog.einverne.info/post/2022/04/mailcow-email-server.html" target="_blank" rel="noopener">使用 Mailcow 自建邮件服务器</a></li><li><a href="https://editor.leonh.space/2023/caddy/" target="_blank" rel="noopener">Caddy Server 簡介</a></li><li><a href="https://blog.wu-boy.com/2017/11/migrate-nginx-to-caddy/" target="_blank" rel="noopener">從 Nginx 換到 Caddy</a></li><li><a href="https://github.com/caddyserver/examples" target="_blank" rel="noopener">caddyserver examples</a></li><li><a href="https://net.nthu.edu.tw/netsys/security:fail2ban" target="_blank" rel="noopener">Fail2ban</a></li><li><a href="https://www.myfreax.com/install-configure-fail2ban-on-ubuntu-20-04/" target="_blank" rel="noopener">如何在Ubuntu 20.04上安装和配置Fail2ban</a></li></ol><hr><p>medium 文章連結：<a href="https://riverye.medium.com/1d80c469efec" target="_blank" rel="noopener">https://riverye.medium.com/1d80c469efec</a><br>本文同步發布於 <a href="https://riverye.com/2023/08/15/How-to-Build-a-Mail-Server-with-Mailcow-on-Ubuntu-Server/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>這篇是紀錄如何自行架設郵件伺服器 (Mail Server) 的過程，使用開源的 mailcow 建置</p>
<h2><span id="說明">說明</span></h2>
<p>當初在找開源的 Emai
</summary>
<category term="教學文" scheme="https://riverye.com/categories/%E6%95%99%E5%AD%B8%E6%96%87/"/>
<category term="Ubuntu" scheme="https://riverye.com/tags/Ubuntu/"/>
<category term="mailcow" scheme="https://riverye.com/tags/mailcow/"/>
</entry>
<entry>
<title>Ubuntu Server 安裝教學</title>
<link href="https://riverye.com/2023/07/29/Ubuntu-Server-Installation-Tutorial/"/>
<id>https://riverye.com/2023/07/29/Ubuntu-Server-Installation-Tutorial/</id>
<published>2023-07-28T16:00:00.000Z</published>
<updated>2023-07-29T14:48:46.275Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>近期自己架設郵件伺服器 (Mail Server) 上線了，是在本地端透過 Ubuntu Server 建置，會將整個過程拆成幾篇文章記錄下，這篇是紀錄如何安裝 Ubuntu Server</p><h2><span id="說明">說明</span></h2><ol><li>到 <a href="https://ubuntu.com/download/server" target="_blank" rel="noopener">Ubuntu 官方下載 ISO</a>，我是選 <code>Ubuntu Server 22.04.2 LTS</code> ，LTS 是長期支援版本，長期支援（LTS）版本每兩年釋出一次。普通版本一般只支援 9 個月，但 LTS 版本一般能提供 5 年的支援。</li><li>為方便截圖安裝步驟，是在 Windows 作業系統上，以 VirtualBox 虛擬機器軟體來安裝作業系統</li><li>將開機順序 USB/DVD 設定成第一順位， VirtualBox 中則是掛載 ISO 檔案</li></ol><h2><span id="安裝步驟">安裝步驟</span></h2><ol><li><p>選第一個<br><img src="Ubuntu_Server_Installation_Tutorial_001.png" alt></p></li><li><p>語系選 English<br><img src="Ubuntu_Server_Installation_Tutorial_002.png" alt></p></li><li><p>選「Continue without updating」<br><img src="Ubuntu_Server_Installation_Tutorial_003.png" alt></p></li><li><p>選「Done」<br><img src="Ubuntu_Server_Installation_Tutorial_004.png" alt></p></li><li><p>依需求選<br><img src="Ubuntu_Server_Installation_Tutorial_005.png" alt></p></li><li><p>由於路由器 (Router) 是設定 DHCP ，會直接分配一個區域網路的 IP ，為避免之後 IP 跑掉，建議可以改成固定 IP<br><img src="Ubuntu_Server_Installation_Tutorial_006.png" alt></p></li><li><p>選「Edit IPv4」<br><img src="Ubuntu_Server_Installation_Tutorial_007.png" alt></p></li><li><p>設置如下 (請依照需求調整)<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">Subnet: 192.168.1.0/24</span><br><span class="line">Address: 192.168.1.126</span><br><span class="line">Gateway: 192.168.1.1</span><br><span class="line">Name Server: 1.1.1.1,1.0.0.1</span><br></pre></td></tr></table></figure></p></li></ol><p><img src="Ubuntu_Server_Installation_Tutorial_008.png" alt></p><ol start="9"><li><p>確認能正常抓到設定後，選「Done」<br><img src="Ubuntu_Server_Installation_Tutorial_009.png" alt></p></li><li><p>選「Done」<br><img src="Ubuntu_Server_Installation_Tutorial_010.png" alt></p></li><li><p>電腦有連網路的話，確認能抓到後，選「Done」<br><img src="Ubuntu_Server_Installation_Tutorial_011.png" alt></p></li><li><p>對 LVM 不熟的話，建議取消打勾，然後選「Done」<br><img src="Ubuntu_Server_Installation_Tutorial_012.png" alt></p></li><li><p>這邊將 VirtualBox 配置的 25 GB 容量全給<br><img src="Ubuntu_Server_Installation_Tutorial_013.png" alt></p></li><li><p>選「Continue」(會把硬碟格式化)<br><img src="Ubuntu_Server_Installation_Tutorial_014.png" alt></p></li><li><p>設定名稱、密碼<br><img src="Ubuntu_Server_Installation_Tutorial_015.png" alt></p></li><li><p>選「Continue」<br><img src="Ubuntu_Server_Installation_Tutorial_016.png" alt></p></li><li><p>勾選安裝「Install OpenSSH server」<br><img src="Ubuntu_Server_Installation_Tutorial_017.png" alt></p></li><li><p>依需求選，選「Done」之後就會開始安裝，會需要一些時間<br><img src="Ubuntu_Server_Installation_Tutorial_018.png" alt></p></li><li><p>安裝完成後，會看到「Reboot Now」的選項<br><img src="Ubuntu_Server_Installation_Tutorial_019.png" alt></p></li><li><p>看到這畫面時，先把 USB/DVD 退出，然後按「Enter」鍵<br><img src="Ubuntu_Server_Installation_Tutorial_020.png" alt></p></li><li><p>等待下，便會看到此畫面<br><img src="Ubuntu_Server_Installation_Tutorial_021.png" alt></p></li><li><p>輸入剛才建立的 username 及密碼，便會看到登入成功的畫面<br><img src="Ubuntu_Server_Installation_Tutorial_022.png" alt></p></li></ol><h2><span id="ssh-連線">SSH 連線</span></h2><p>以 PowerShell 示範，用 SSH 連線到剛才建立的 Ubuntu Server<figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> ssh demo@192.168.1.126</span></span><br></pre></td></tr></table></figure></p><p><img src="demo_ssh.png" alt></p><hr><h2><span id="安裝完成後的設定">安裝完成後的設定</span></h2><h3><span id="習慣先將系統更新到最新">習慣先將系統更新到最新</span></h3><p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> apt update -y && apt upgrade -y</span></span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 等上面指令跑完後，會有提示訊息，接著重開 server</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> reboot</span></span><br></pre></td></tr></table></figure></p><h3><span id="安裝一些常用的套件">安裝一些常用的套件</span></h3><p>習慣使用 <code>htop</code> 及 <code>ripgrep</code><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> apt install htop ripgrep -y</span></span><br></pre></td></tr></table></figure></p><h3><span id="改作業系統的時區">改作業系統的時區</span></h3><p>預設是 <code>Etc/UTC (UTC, +0000)</code> 時區，改成 Asia/Taipei 時區<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> timedatectl</span></span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 改成 Asia/Taipei 時區</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> timedatectl <span class="built_in">set</span>-timezone Asia/Taipei</span></span><br></pre></td></tr></table></figure></p><p><img src="timedatectl.png" alt></p><h3><span id="調整-ssh-設定">調整 SSH 設定</span></h3><p>為避免有人想要惡意連線、猜密碼，除了更改 SSH 預設 22 port 之外，也要改成不允許輸入密碼 SSH 連線，只有 SSH public key 被加入的可以連線<br>另外可以安裝 Fail2ban 來 ban 惡意連線的 IP<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> vim /etc/ssh/sshd_config</span></span><br></pre></td></tr></table></figure></p><p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># /etc/ssh/sshd_config</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># before</span></span><br><span class="line"><span class="comment">#Port 22</span></span><br><span class="line"><span class="comment">#PasswordAuthentication yes</span></span><br><span class="line"><span class="comment">#PermitEmptyPasswords no</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># after, 移除前面 #, 調整設定</span></span><br><span class="line">Port 9527</span><br><span class="line">PasswordAuthentication no</span><br><span class="line">PermitEmptyPasswords no</span><br></pre></td></tr></table></figure></p><p>上述的 <code>/etc/ssh/sshd_config</code> 檔案中，可將 <code>Include /etc/ssh/sshd_config.d/*.conf</code> 這行註解<br>或者更改檔案 <code>/etc/ssh/sshd_config.d/50-cloud-init.conf</code> 中的設定，二擇一即可<br>改成以下<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># /etc/ssh/sshd_config.d/50-cloud-init.conf</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># before</span></span><br><span class="line">PasswordAuthentication yes</span><br><span class="line"></span><br><span class="line"><span class="comment"># after</span></span><br><span class="line">PasswordAuthentication no</span><br></pre></td></tr></table></figure></p><p>改好設定後，要重啟服務，才會生效<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">$</span><span class="bash"> service ssh restart</span></span><br></pre></td></tr></table></figure></p><p>接著用區域網路內其他電腦，嘗試連線看看<br><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash"> 使用 SSH 預設 22 port 連線</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> ssh demo@192.168.1.126</span></span><br><span class="line">ssh: connect to host 192.168.1.126 port 22: Connection refused</span><br><span class="line"></span><br><span class="line"><span class="meta">#</span><span class="bash"> 使用剛設定的 9527 port 連線</span></span><br><span class="line"><span class="meta">$</span><span class="bash"> ssh demo@192.168.1.126 -p 9527</span></span><br><span class="line">demo@192.168.1.126: Permission denied (publickey).</span><br></pre></td></tr></table></figure></p><hr><h2><span id="小結">小結</span></h2><p>在設定 <code>sshd_config</code> 時，沒發現 <code>Include /etc/ssh/sshd_config.d/*.conf</code> 這行，很困惑怎設定沒生效，也重開機過，感謝 Michael 協助，才發現小丑是我自己 xDD</p><hr><h2><span id="參考文件">參考文件</span></h2><p><a href="https://indie.tw/tutorials/ubuntu-server/" target="_blank" rel="noopener">實戰 Ubuntu Server 安裝</a></p><hr><p>medium 文章連結：<a href="https://riverye.medium.com/19d070c1623c" target="_blank" rel="noopener">https://riverye.medium.com/19d070c1623c</a><br>本文同步發布於 <a href="https://riverye.com/2023/07/29/Ubuntu-Server-Installation-Tutorial/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>近期自己架設郵件伺服器 (Mail Server) 上線了，是在本地端透過 Ubuntu Server 建置，會將整個過程拆成幾篇文章記錄下，這篇是紀錄如何安裝 Ubuntu Server</p>
<h2><
</summary>
<category term="教學文" scheme="https://riverye.com/categories/%E6%95%99%E5%AD%B8%E6%96%87/"/>
<category term="Ubuntu" scheme="https://riverye.com/tags/Ubuntu/"/>
</entry>
<entry>
<title>如何使用 Postman 測試 API</title>
<link href="https://riverye.com/2022/12/31/How-to-Test-APIs-with-Postman/"/>
<id>https://riverye.com/2022/12/31/How-to-Test-APIs-with-Postman/</id>
<published>2022-12-30T16:00:00.000Z</published>
<updated>2022-12-31T16:12:37.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>不論前後端在處理或串接 API 時，除了在瀏覽器測試外，透過 <a href="https://www.postman.com/" target="_blank" rel="noopener">Postman</a> 進行測試是非常地方便，這篇主要是紀錄如何使用，之後有人問時，就能請對方直接看這篇了 xd</p><h2><span id="說明">說明</span></h2><p>Windows 與 macOS 皆能到 <a href="https://www.postman.com/downloads/" target="_blank" rel="noopener">Postman</a> 官方網站下載，若不想安裝，也能使用 Postman 提供的 <a href="https://web.postman.co/" target="_blank" rel="noopener">web 版</a>，使用 web 版需要先註冊才能使用，介面大同小異，以下會以 macOS 操作為主</p><p>Postman 有免費版與付費版，我自己是用免費版，需求已經夠我使用，若需求比較多，免費版 Postman 已經無法滿足需求，但又不想付費時，可以考慮用 <a href="https://hoppscotch.io/" target="_blank" rel="noopener">Hoppscotch</a> (原名是 <a href="https://postwoman.io/" target="_blank" rel="noopener">Postwoman</a>)，Hoppscotch 是<a href="https://github.com/hoppscotch/hoppscotch" target="_blank" rel="noopener">開源軟體</a>，介面操作上與 Postman 也很相似，也有提供 web 版可以操作</p><p>note: 類似替代軟體蠻多的，就不多做贅述</p><h2><span id="圖文教學">圖文教學</span></h2><p>對我來說，動手做學習效果最好，以下直接提供 HTTP GET、POST 的範例，以及遇到要登入的網站時，要如何用 Postman 打 API</p><h3><span id="示範用-postman-http-get-不用登入的網站">示範用 Postman HTTP GET - 不用登入的網站</span></h3><p>以 <a href="https://echo.hoppscotch.io" target="_blank" rel="noopener">https://echo.hoppscotch.io</a> 為例<br>測試完後，點 <code></></code> icon 並選想要的程式語言範例 code (這超好用)<br><img src="HTTP_GET_%E7%AF%84%E4%BE%8B-%E4%B8%8D%E7%94%A8%E7%99%BB%E5%85%A5%E7%9A%84%E7%B6%B2%E7%AB%99.png" alt></p><h3><span id="示範用-postman-http-post-要登入的網站">示範用 Postman HTTP POST - 要登入的網站</span></h3><p>以 <a href="https://data.gov.tw/" target="_blank" rel="noopener">政府資料開放平臺</a> 的 <a href="https://data.gov.tw/user/subscriptions" target="_blank" rel="noopener">訂閱資料集清單</a> 為例</p><p><div class="video-container"><iframe src="//www.youtube.com/embed/oHzn9hoq7QE" frameborder="0" allowfullscreen></iframe></div>在 Chrome 的開發者工具 (F12) 中，透過 Network 可以觀察網站打了哪些 endpoint<br><img src="Chrome%E7%9A%84Network.png" alt></p><h3><span id="示範打用-swagger-建立的網站">示範打用 Swagger 建立的網站</span></h3><p>簡易說明: <a href="https://swagger.io/" target="_blank" rel="noopener">Swagger</a> 可用來自動產生 API 文件，蠻多提供 API 的網站是用 Swagger 建立的，看到下圖這類型風格的網站，很高機率都是用 Swagger 建立的</p><p>圖中範例的網址: <a href="https://data.gcis.nat.gov.tw/resources/swagger/" target="_blank" rel="noopener">商工行政資料開放平臺 data.gcis.nat.gov.tw</a><img src="Swagger_example.png" alt></p><p>這邊以 <a href="https://www.cyberbiz.io/" target="_blank" rel="noopener">CYBERBIZ</a> 的 API 為例，可 Google 搜尋「CYBERBIZ API」即可找到<a href="https://www.cyberbiz.io/support/?p=20739" target="_blank" rel="noopener">相關文件與連結</a></p><h3><span id="示範用-postman-http-get-打-cyberbiz-的-api">示範用 Postman HTTP GET 打 CYBERBIZ 的 API</span></h3><p>note: 範例中的網址與相關功能，若 CYBERBIZ 有更動或失效，則不另外更新影片<br><div class="video-container"><iframe src="//www.youtube.com/embed/PGVPvpBPLi0" frameborder="0" allowfullscreen></iframe></div></p><h3><span id="示範用-postman-http-post-打-cyberbiz-的-api">示範用 Postman HTTP POST 打 CYBERBIZ 的 API</span></h3><p>note: 範例中的網址與相關功能，若 CYBERBIZ 有更動或失效，則不另外更新影片<br><div class="video-container"><iframe src="//www.youtube.com/embed/cJXictjQotU" frameborder="0" allowfullscreen></iframe></div></p><p>以下是自己寫的 Ruby Class，方便可以用 POST (GET 就不寫了 xd)<br><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">require</span> <span class="string">'uri'</span></span><br><span class="line"><span class="keyword">require</span> <span class="string">'net/http'</span></span><br><span class="line"><span class="keyword">require</span> <span class="string">'digest'</span></span><br><span class="line"><span class="keyword">require</span> <span class="string">'base64'</span></span><br><span class="line"><span class="keyword">require</span> <span class="string">'openssl'</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">CyberbizApi</span></span></span><br><span class="line"> <span class="keyword">attr_reader</span> <span class="symbol">:username</span>, <span class="symbol">:secret</span></span><br><span class="line"></span><br><span class="line"> URL = <span class="string">'https://api.cyberbiz.co/v1/'</span>.freeze</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">initialize</span></span></span><br><span class="line"> @username = <span class="string">'your_username'</span></span><br><span class="line"> @secret = <span class="string">'your_secret'</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">post</span><span class="params">(path, data)</span></span></span><br><span class="line"> send_request(<span class="symbol">:post</span>, path, data)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> private</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">send_request</span><span class="params">(request_method, path, data)</span></span></span><br><span class="line"> uri = URI(URL + path)</span><br><span class="line"> uri.query = URI.encode_www_form(data)</span><br><span class="line"> http = init_http(uri)</span><br><span class="line"> request = build_request(request_method, uri, data)</span><br><span class="line"> send_request_and_return_response(http, request)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">init_http</span><span class="params">(uri)</span></span></span><br><span class="line"> http = Net::HTTP.new(uri.host, uri.port)</span><br><span class="line"> http.use_ssl = uri.scheme == <span class="string">'https'</span></span><br><span class="line"> http</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">build_request</span><span class="params">(request_method, uri, data)</span></span></span><br><span class="line"> request =</span><br><span class="line"> <span class="keyword">case</span> request_method</span><br><span class="line"> <span class="keyword">when</span> <span class="symbol">:get</span></span><br><span class="line"> Net::HTTP::Get.new(uri.request_uri)</span><br><span class="line"> <span class="keyword">when</span> <span class="symbol">:post</span></span><br><span class="line"> Net::HTTP::Post.new(uri.request_uri)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> setting_request(request, uri, data)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">setting_request</span><span class="params">(request, uri, data)</span></span></span><br><span class="line"> request.content_type = <span class="string">'application/x-www-form-urlencoded'</span></span><br><span class="line"> request[<span class="string">'x-date'</span>] = Time.now.gmtime.strftime(<span class="string">'%a, %d %b %Y %T GMT'</span>)</span><br><span class="line"> request.body = uri.query</span><br><span class="line"> digest = Base64.strict_encode64(Digest::SHA256.digest(request.body))</span><br><span class="line"> request[<span class="string">'digest'</span>] = <span class="string">"SHA-256=<span class="subst">#{digest}</span>"</span></span><br><span class="line"> signature = generate_signature(request)</span><br><span class="line"> request[<span class="string">"authorization"</span>] = <span class="string">"hmac username=\"<span class="subst">#{username}</span>\", "</span>\</span><br><span class="line"> <span class="string">"algorithm=\"hmac-sha256\", "</span>\</span><br><span class="line"> <span class="string">"headers=\"x-date request-line digest\", "</span>\</span><br><span class="line"> <span class="string">"signature=\"<span class="subst">#{signature}</span>\""</span></span><br><span class="line"> request</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">generate_signature</span><span class="params">(request)</span></span></span><br><span class="line"> x_date = request.to_hash[<span class="string">'x-date'</span>].first</span><br><span class="line"> digest = request.to_hash[<span class="string">'digest'</span>].first</span><br><span class="line"> sig_str = <span class="string">"x-date: <span class="subst">#{x_date}</span>\n<span class="subst">#{request.method}</span> <span class="subst">#{request.path}</span> HTTP/1.1\ndigest: <span class="subst">#{digest}</span>"</span></span><br><span class="line"> sha256_digest = OpenSSL::Digest.new(<span class="string">'sha256'</span>)</span><br><span class="line"> hmac_digest = OpenSSL::HMAC.digest(sha256_digest, secret, sig_str)</span><br><span class="line"> Base64.strict_encode64(hmac_digest)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">send_request_and_return_response</span><span class="params">(http, request)</span></span></span><br><span class="line"> response = http.request(request)</span><br><span class="line"> response.body</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><p>測試打「新增商品」<br><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">data = {</span><br><span class="line"> <span class="symbol">title:</span> <span class="string">"test"</span>,</span><br><span class="line"> <span class="symbol">handle:</span> <span class="string">"hello"</span>,</span><br><span class="line"> <span class="symbol">published:</span> <span class="literal">true</span>,</span><br><span class="line"> <span class="symbol">price:</span> <span class="number">666</span>,</span><br><span class="line"> <span class="symbol">tax_type_id:</span> <span class="string">"inclusive_tax"</span>,</span><br><span class="line">}</span><br><span class="line">response = CyberbizApi.new.post(<span class="string">'products'</span>, data)</span><br></pre></td></tr></table></figure></p><h3><span id="示範用-postman-打-line-notify-api">示範用 Postman 打 LINE Notify API</span></h3><p>可參考之前寫的這篇<a href="https://riverye.com/2022/01/30/line-notify/#6-%E7%94%A8-postman-%E6%B8%AC%E8%A9%A6%E7%9C%8B%E7%9C%8B-1">用 Ruby 將訊息傳到 LINE 群組 (LINE Notify API)</a></p><h3><span id="若遇到需要夾帶上傳檔案時">若遇到需要夾帶(上傳)檔案時</span></h3><p>Postman 有支援夾帶檔案<img src="Upload_file.png" alt></p><h2><span id="小結">小結</span></h2><p>若對 Postman 不熟建議可以看上面的 Youtube 影片範例，可以到 Youtube 的網站觀看，都已經把時間戳寫好了，全部看完不用 5 分鐘</p><p>千萬不要相信前端傳送的資料，通常後端會再次驗證資料，因為前端傳的資料能輕易被修改、自定義多傳參數到後端</p><hr><p>medium 文章連結：<a href="https://link.medium.com/ZdX8mEXVcwb" target="_blank" rel="noopener">https://link.medium.com/ZdX8mEXVcwb</a><br>本文同步發布於 <a href="https://riverye.com/2022/12/31/How-to-Test-APIs-with-Postman/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>不論前後端在處理或串接 API 時，除了在瀏覽器測試外，透過 <a href="https://www.postman.com/" target="_blank" rel="noopener">Postman
</summary>
<category term="教學文" scheme="https://riverye.com/categories/%E6%95%99%E5%AD%B8%E6%96%87/"/>
<category term="Postman" scheme="https://riverye.com/tags/Postman/"/>
<category term="API" scheme="https://riverye.com/tags/API/"/>
</entry>
<entry>
<title>如何擷取電腦、手機的封包 (圖文教學)</title>
<link href="https://riverye.com/2022/12/20/Capturing-Packets-on-a-Computer-and-Mobile-Phone/"/>
<id>https://riverye.com/2022/12/20/Capturing-Packets-on-a-Computer-and-Mobile-Phone/</id>
<published>2022-12-19T16:00:00.000Z</published>
<updated>2022-12-20T15:43:42.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>透過網路封包擷取工具，可以擷取到 Windows、Mac、Mobile Phone 等不同裝置上的封包，自己用過 <a href="https://zh.wikipedia.org/zh-tw/Burp_suite" target="_blank" rel="noopener">Burp Suite</a>、<a href="https://zh.wikipedia.org/zh-tw/%E6%9F%A5%E7%88%BE%E6%96%AF%E4%BB%A3%E7%90%86" target="_blank" rel="noopener">Charles</a>、<a href="https://zh.wikipedia.org/zh-tw/Fiddler" target="_blank" rel="noopener">Fiddler</a> 等這幾套軟體，以自己需求來說，<a href="https://www.charlesproxy.com/" target="_blank" rel="noopener">Charles</a> 就夠用了，因此這篇會以這套軟體操作介紹為主。</p><h2><span id="說明">說明</span></h2><p>有在 macOS Ventura 13.0.1 、iOS 16.2 、Windows 10、Android 交叉測試過，確認是可以擷取到封包，由於 Windows 與 macOS 操作大同小異，以及本身是用 iPhone ，故以下說明與截圖會是 macOS 和 iPhone 為主</p><p>因目前許多網站和伺服器是以 HTTPS 來加密傳輸資料，若需要解析加密的資料，可用 Charles 內建提供的憑證來處理，就能看到加密前的資料，若有疑慮可用其它軟體以及另外準備憑證處理</p><p>開始前請先到<a href="https://www.charlesproxy.com/download/" target="_blank" rel="noopener">官網下載安裝 Charles</a></p><h2><span id="免費版限制">免費版限制</span></h2><p>雖說 Charles 是付費軟體，但可以免費使用，限制如下:</p><ol><li><p>開啟程式時，需要等待 10 秒<img src="wait_ten_secondes.png" alt="等待10秒的畫面"></p></li><li><p>每次僅能使用 30 分鐘，超過需要重啟程式<img src="usage_restrictions.png" alt="每使用30分鐘會跳一次訊息"></p></li></ol><p>note: 上述限制已夠我使用，故不影響 (不確定未來政策是否會調整)</p><hr><h2><span id="macos-設定教學">macOS 設定教學</span></h2><h3><span id="影片版教學">影片版教學</span></h3><p><div class="video-container"><iframe src="//www.youtube.com/embed/EIf9zmPCJwE" frameborder="0" allowfullscreen></iframe></div></p><h3><span id="圖文教學">圖文教學</span></h3><ol><li><p>安裝憑證 (Certificate)<br>在 Charles 的 Help → SSL Proxying<br>可選擇直接安裝 (Install Charles Root Certificate)，或先另存憑證 (Save Charles Root Certificate)，在手動匯入<br>這邊以手動匯入為例<img src="charles_settings_01.png" alt="Save Charles Root Certificate"></p></li><li><p>手動匯入憑證 (直接安裝者，可略過這步驟)<br>在 macOS 的 應用程式 → 工具程式 → 鑰匙圈存取<br>將另存的憑證手動匯入，並對該憑證點滑鼠右鍵的「取得資訊」，在「信任」中，改成「永遠信任」<br><img src="charles_settings_02.png" alt="Setting Certificate"></p></li><li><p>設定 Proxy<br>在 Charles 的 Proxy → SSL Proxying Settings → SSL Proxying<br>3-1. 勾選「Enable SSL Proxying」<br>3-2. 設定所有 Host 以及 Port 都要解析 (Empty fields match all values.)<br>3-3. 「macOS Proxy」也要開著<br><img src="charles_settings_03.png" alt="Setting Proxy"></p></li><li><p>開始紀錄 (Start Recording)<br>以 Visual Studio Code 為例<img src="charles_settings_04.png" alt="Start Recording"></p></li></ol><hr><h2><span id="iphone-設定教學">iPhone 設定教學</span></h2><h3><span id="影片版教學">影片版教學</span></h3><p>有錄影，但有些資料較敏感(ex: 手機序號等)，且懶得後製影片，故不放 xd</p><h3><span id="圖文教學">圖文教學</span></h3><ol><li><p>設定 HTTP Proxy Port<br>Proxy → Proxy Settings → Proxies → Port: 8888 → OK<br><img src="iphone_settings_01.png" alt></p></li><li><p>查看電腦的區域網路 IP<br>在 Charles 的 Proxy → Local IP Addresses<br><img src="iphone_settings_02.png" alt></p></li><li><p>Install Charles Root Certificate in iOS Simulators<br><img src="iphone_settings_03.png" alt></p></li><li><p>Install Charles Root Certificate on a Mobile Device or Remote Browser<br>這邊會提醒手機的代理伺服器以及要連哪個網站才能下載憑證<br><img src="iphone_settings_04.png" alt></p></li><li><p>需確保電腦與手機在同個區域網路<br><img src="iphone_settings_05.png" alt></p></li><li><p>手機設定代理伺服器<br>手機的設定 → 一般 → Wi-Fi → 點「i」的 icon 圖案 → 設定代理伺服器<img src="iphone_settings_06.png" alt></p></li><li><p>用 Safari 開啟 <a href="chls.pro/ssh">chls.pro/ssh</a>，並下載憑證<br><img src="iphone_settings_07.png" alt></p></li><li><p>手機安裝憑證<br>手機的設定 → 已下載描述檔 → 安裝 (安裝後，可在這裡查看: 設定 → 一般 → VPN 與 裝置管)<br>測完，可在這移除<br><img src="iphone_settings_08.png" alt></p></li><li><p>手動開啟憑證 <strong>(開啟後，建議先把手機的 Wi-Fi 先關閉再開啟)</strong><br>手機的設定 → 一般 → 關於本機 → 憑證信任設定 → 測試時記得開啟<br><img src="iphone_settings_09.png" alt></p></li><li><p>以 GitHub App 為例<br><img src="iphone_settings_10.png" alt></p></li></ol><hr><h2><span id="小結">小結</span></h2><p>會有這篇是為了解決自己的痛點，像是買的除濕機提供的 App 只能設定 7 組開關時間，但我想要每天早晚指定時間都自動開啟 1 小時，因超過 App 提供 7 組的設定上限，於是乎有了解析手機 App 封包的想法，觀察打的是哪隻 API 並用 Postman 測試，最後寫成 iPhone 的捷徑與自動化，還能用 Siri 語音控制，進而實現自動化解決這問題</p><p>其實我 3 個多月前就已經完成上述的事情，發現在寫這篇文章時，有些細節都忘了，趁還有動力時，趕緊做個紀錄，免得哪天又忘記時，也能當筆記回顧</p><p>其他類似的軟體蠻多的，像是知名的 <a href="https://zh.wikipedia.org/zh-tw/Wireshark" target="_blank" rel="noopener">Wireshark</a> 等軟體，這邊就不多贅述了</p><hr><p>medium 文章連結：<a href="https://link.medium.com/3SaxKRlRUvb" target="_blank" rel="noopener">https://link.medium.com/3SaxKRlRUvb</a><br>本文同步發布於 <a href="https://riverye.com/2022/12/20/Capturing-Packets-on-a-Computer-and-Mobile-Phone/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>透過網路封包擷取工具，可以擷取到 Windows、Mac、Mobile Phone 等不同裝置上的封包，自己用過 <a href="https://zh.wikipedia.org/zh-tw/Burp_su
</summary>
<category term="教學文" scheme="https://riverye.com/categories/%E6%95%99%E5%AD%B8%E6%96%87/"/>
<category term="抓包軟體" scheme="https://riverye.com/tags/%E6%8A%93%E5%8C%85%E8%BB%9F%E9%AB%94/"/>
<category term="網路封包擷取" scheme="https://riverye.com/tags/%E7%B6%B2%E8%B7%AF%E5%B0%81%E5%8C%85%E6%93%B7%E5%8F%96/"/>
<category term="Charles" scheme="https://riverye.com/tags/Charles/"/>
</entry>
<entry>
<title>在 Ruby 中用 Tor 打 API：如何隱藏真實 IP</title>
<link href="https://riverye.com/2022/12/17/Protecting-Your-Real-IP-When-Hitting-APIs-with-Ruby-and-Tor/"/>
<id>https://riverye.com/2022/12/17/Protecting-Your-Real-IP-When-Hitting-APIs-with-Ruby-and-Tor/</id>
<published>2022-12-16T16:00:00.000Z</published>
<updated>2022-12-31T07:02:44.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>用 Ruby 在打 API 時，若不想讓人知道請求的 IP (request IP)，可以搭配用 Tor 來實踐。<br><a href="https://zh.m.wikipedia.org/zh-tw/Tor" target="_blank" rel="noopener">Tor (The Onion Router)</a> 是一個用於匿名通訊的網絡，可以用來瀏覽網頁、傳輸數據和通訊。<br>本文將介紹如何在 Ruby 中使用 Tor 進行匿名瀏覽和通訊。</p><h2><span id="說明">說明</span></h2><p>這邊以 Ruby 3.1.2 為範例，並提供 2 種方法作為參考</p><h1><span id="方法一">方法一</span></h1><p>用別人寫好的 Docker image <a href="https://github.com/mattes/rotating-proxy" target="_blank" rel="noopener">docker-rotating-proxy</a><br>note: 我個人比較習慣用這個</p><p>照著官方文件下載與執行 Docker container<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">docker run -d -p 5566:5566 -p 4444:4444 --env tors=25 mattes/rotating-proxy</span><br></pre></td></tr></table></figure></p><p>在 Command-line 測試<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">curl --proxy 127.0.0.1:5566 https://api.my-ip.io/ip</span><br></pre></td></tr></table></figure></p><p>以下是自己寫的 Ruby Class，方便可以用 GET 或 POST<figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">require</span> <span class="string">'net/http'</span></span><br><span class="line"><span class="keyword">require</span> <span class="string">'json'</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">TorHTTP</span></span></span><br><span class="line"> <span class="keyword">attr_reader</span> <span class="symbol">:proxy_address</span>, <span class="symbol">:proxy_port</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 在這裡指定 Tor 的 SOCKS 代理的位置</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">initialize</span><span class="params">(<span class="symbol">proxy_address:</span> <span class="string">'127.0.0.1'</span>, <span class="symbol">proxy_port:</span> <span class="number">5566</span>)</span></span></span><br><span class="line"> @proxy_address = proxy_address</span><br><span class="line"> @proxy_port = proxy_port</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">get</span><span class="params">(url, <span class="symbol">data:</span> <span class="literal">nil</span>)</span></span></span><br><span class="line"> send_request(<span class="symbol">:get</span>, url, data)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">post</span><span class="params">(url, <span class="symbol">data:</span> <span class="literal">nil</span>)</span></span></span><br><span class="line"> send_request(<span class="symbol">:post</span>, url, data)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> private</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">send_request</span><span class="params">(request_method, url, data)</span></span></span><br><span class="line"> uri = URI(url)</span><br><span class="line"> http = init_http(uri)</span><br><span class="line"> request = build_request(request_method, uri)</span><br><span class="line"> add_request_body(request, data) <span class="keyword">if</span> data</span><br><span class="line"> send_request_and_return_response(http, request)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">init_http</span><span class="params">(uri)</span></span></span><br><span class="line"> http = Net::HTTP.new(uri.host, uri.port, proxy_address, proxy_port)</span><br><span class="line"> http.use_ssl = uri.scheme == <span class="string">'https'</span></span><br><span class="line"> http.open_timeout = <span class="number">10</span></span><br><span class="line"> http.continue_timeout = <span class="number">10</span></span><br><span class="line"> http</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">build_request</span><span class="params">(request_method, uri)</span></span></span><br><span class="line"> <span class="keyword">case</span> request_method</span><br><span class="line"> <span class="keyword">when</span> <span class="symbol">:get</span></span><br><span class="line"> Net::HTTP::Get.new(uri.request_uri)</span><br><span class="line"> <span class="keyword">when</span> <span class="symbol">:post</span></span><br><span class="line"> Net::HTTP::Post.new(uri.request_uri)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">add_request_body</span><span class="params">(request, data)</span></span></span><br><span class="line"> request.body = data.to_json</span><br><span class="line"> request.content_type = <span class="string">'application/json'</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">send_request_and_return_response</span><span class="params">(http, request)</span></span></span><br><span class="line"> response = http.request(request)</span><br><span class="line"> response.body</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><p>測試打查詢 IP 看 IP 是否會變<figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">TorHTTP.new.get(<span class="string">'https://ifconfig.me'</span>)</span><br></pre></td></tr></table></figure></p><p><img src="TiSiTOK.png" alt></p><h3><span id="改成打-post-測試">改成打 POST 測試</span></h3><p>可以參考之前寫的這篇 <a href="https://riverye.com/2020/09/20/Day15-%E5%A6%82%E4%BD%95%E7%9C%8B%E8%87%AA%E5%B7%B1%E6%89%93%E5%87%BA%E5%8E%BB%E7%9A%84-request-%E5%AE%8C%E6%95%B4%E8%B3%87%E8%A8%8A%EF%BC%8C%E4%BB%A5-PostBin-%E5%92%8C-Webhook-site-%E7%82%BA%E4%BE%8B/">如何看自己打出去的 request 完整資訊，以 PostBin 和 Webhook.site 為例</a><br>這邊以 PostBin 為例<figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">data = { <span class="symbol">hello:</span> <span class="string">'world'</span> }</span><br><span class="line">TorHTTP.new.post(<span class="string">'https://www.toptal.com/developers/postbin/1671278873904-9991861544549'</span>, <span class="symbol">data:</span> data)</span><br></pre></td></tr></table></figure></p><p><img src="8Xng2AD.png" alt></p><h1><span id="方法二">方法二</span></h1><ol><li>用別人寫好的 Docker image <a href="https://github.com/PeterDaveHello/tor-socks-proxy" target="_blank" rel="noopener">Tor-socks-proxy</a></li><li>除了上述步驟外，也要安裝 <a href="https://rubygems.org/gems/socksify" target="_blank" rel="noopener">socksify gem</a><br><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 也有其他方式能做到，請看官方文件</span></span><br><span class="line"><span class="comment"># 進入 IRB</span></span><br><span class="line">$ socksify_ruby localhost <span class="number">9150</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">require</span> <span class="string">'http'</span></span><br><span class="line">HTTP.get(<span class="string">'https://ipinfo.io/ip'</span>).to_s</span><br><span class="line"></span><br><span class="line"><span class="comment"># 或跑腳本</span></span><br><span class="line">socksify_ruby localhost <span class="number">9150</span> script.rb</span><br></pre></td></tr></table></figure></li></ol><p>可以發現，它不像方法一的每次 request 的 IP 位址都不同，可看需求使用</p><p><img src="IxWpeAm.png" alt></p><hr><h2><span id="如何查自己-ip-地址">如何查自己 IP 地址</span></h2><p>這邊紀錄幾個比較常用的</p><ol><li><a href="https://ipinfo.io/ip" target="_blank" rel="noopener">https://ipinfo.io/ip</a></li><li><a href="https://ipecho.net/plain" target="_blank" rel="noopener">https://ipecho.net/plain</a></li><li><a href="https://api.my-ip.io/ip" target="_blank" rel="noopener">https://api.my-ip.io/ip</a></li><li><a href="https://ifconfig.me" target="_blank" rel="noopener">https://ifconfig.me</a></li><li><a href="https://ipinfo.tw/ip" target="_blank" rel="noopener">https://ipinfo.tw/ip</a></li><li><a href="https://www.whatismyip.com/" target="_blank" rel="noopener">https://www.whatismyip.com/</a></li><li><a href="https://ipecho.net/plain" target="_blank" rel="noopener">https://ipecho.net/plain</a></li><li><a href="https://check.torproject.org/api/ip" target="_blank" rel="noopener">https://check.torproject.org/api/ip</a></li></ol><h2><span id="小結">小結</span></h2><p>在爬網站資料時，若不想讓人知道實際 IP 時，這是一個還不錯的方法，並租一台機器在雲端上跑，就更難被發現了 XD</p><hr><p>medium 文章連結：<a href="https://link.medium.com/Aw09O7sMPvb" target="_blank" rel="noopener">https://link.medium.com/Aw09O7sMPvb</a><br>本文同步發布於 <a href="https://riverye.com/2022/12/17/Protecting-Your-Real-IP-When-Hitting-APIs-with-Ruby-and-Tor/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>用 Ruby 在打 API 時，若不想讓人知道請求的 IP (request IP)，可以搭配用 Tor 來實踐。<br>
<a href="https://zh.m.wikipedia.org/zh-tw/
</summary>
<category term="教學文" scheme="https://riverye.com/categories/%E6%95%99%E5%AD%B8%E6%96%87/"/>
<category term="Ruby" scheme="https://riverye.com/tags/Ruby/"/>
<category term="Tor" scheme="https://riverye.com/tags/Tor/"/>
</entry>
<entry>
<title>用 Ruby on Rails 幫 Excel 檔案加密、解密和欄位加上超連結、修改字形色彩</title>
<link href="https://riverye.com/2022/01/31/excel-advanced-skills/"/>
<id>https://riverye.com/2022/01/31/excel-advanced-skills/</id>
<published>2022-01-30T16:00:00.000Z</published>
<updated>2022-12-21T14:34:31.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>延續 <a href="https://riverye.com/2021/09/29/Day16-%E5%8C%AF%E5%87%BA-excel-%E6%87%89%E7%94%A8%E7%AF%87/">2021 鐵人賽這篇文章 (Day16 - 匯出 excel-應用篇)</a>，進階介紹如行幫 Excel 檔案加密/解密，還有幫欄位加上超連結、修改字形色彩、增加篩選、凍結第一列..等。</p><h2><span id="說明">說明</span></h2><p>標題已經夠淺白，應該不用多說明吧 (笑</p><p>note: 延續 2021 鐵人賽的 <a href="https://github.com/River-Ye/ironman_13th_2021/pull/6" target="_blank" rel="noopener">repo 中的範例</a></p><h4><span id="demo-用的假資料-excel">demo 用的假資料 Excel</span></h4><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 產一份 demo 用的假資料</span></span><br><span class="line"></span><br><span class="line">xlsx = ShopsExcel::Generator.new.execute</span><br><span class="line">xlsx.use_shared_strings = <span class="literal">true</span></span><br><span class="line">xlsx_file = Rails.root.join(<span class="string">'data/river_demo.xlsx'</span>)</span><br><span class="line">xlsx.serialize(xlsx_file) <span class="comment"># 裡面已經有 4 筆假資料</span></span><br></pre></td></tr></table></figure></p><p><img src="mzeYefq.png" alt></p><h1><span id="excel-加密">Excel 加密</span></h1><h2><span id="使用-secure-spreadsheet-來加密">使用 來加密</span></h2><p><a href="https://github.com/ankane/secure-spreadsheet" target="_blank" rel="noopener">secure-spreadsheet 官方文件</a>很好上手，文字不多，很快就能看完<br>先安裝 <code>npm install -g secure-spreadsheet</code><br>若要針對 XLSX 加密(保護) 的話，CLI example</p><p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">secure-spreadsheet --password secret --input-format xlsx < input.xlsx > output.xlsx</span><br></pre></td></tr></table></figure></p><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># rails console</span></span><br><span class="line"><span class="comment"># 使用 secure-spreadsheet 來加密</span></span><br><span class="line"></span><br><span class="line">input_xlsx_file = Rails.root.join(<span class="string">'data/river_demo.xlsx'</span>)</span><br><span class="line">password = <span class="string">'我是密碼-River'</span></span><br><span class="line">result, status = Open3.capture2(<span class="string">'secure-spreadsheet'</span>, <span class="string">'--password'</span>, password, <span class="string">'--input-format'</span>, <span class="string">'xlsx'</span>, <span class="symbol">stdin_data:</span> File.open(input_xlsx_file))</span><br><span class="line">encrypted_xlsx_file_path = input_xlsx_file.dirname.join(<span class="string">'river_demo_encrypted.xlsx'</span>) <span class="comment"># 加密後的檔案</span></span><br><span class="line">File.write(encrypted_xlsx_file_path, result)</span><br></pre></td></tr></table></figure></p><p><img src="skDVo5x.png" alt></p><p>看完上面後，有沒有很簡單，可惜這套僅支援加密<br>若同時有加密/解密需求的話，可參考以下另一個方法</p><h1><span id="excel-加密解密-另一個方法">Excel 加密/解密 (另一個方法)</span></h1><h2><span id="使用-msoffice-進行加密解密-excel-檔案">使用 進行加密/解密 Excel 檔案</span></h2><p><a href="https://github.com/herumi/msoffice" target="_blank" rel="noopener">msoffice 官方文件</a> 雖然寫是在 64-bit Windows 執行，但也有支援 Linux 喔!!</p><p>note: 安裝方法請參照<a href="https://github.com/herumi/msoffice" target="_blank" rel="noopener">官方文件說明</a></p><p>在 macOS 上實測加密/解密，是沒問題的!!</p><p>以下示範如何解密 Excel 檔案，就不再贅述如何加密</p><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># rails console</span></span><br><span class="line"><span class="comment"># 假設 msoffice path 與 repo path 是同目錄名 (dirname)</span></span><br><span class="line"></span><br><span class="line">msoffice = Pathname.new(Dir.pwd).dirname.join(<span class="string">'msoffice'</span>, <span class="string">'bin'</span>, <span class="string">'msoffice-crypt.exe'</span>)</span><br><span class="line"><span class="comment"># or</span></span><br><span class="line">msoffice = Pathname.new(Dir.pwd).dirname.join(<span class="string">'msoffice/bin/msoffice-crypt.exe'</span>)</span><br><span class="line"></span><br><span class="line">input_xlsx_file = Rails.root.join(<span class="string">'data/river_demo.xlsx'</span>)</span><br><span class="line">encrypted_xlsx_file_path = input_xlsx_file.dirname.join(<span class="string">'river_demo_encrypted.xlsx'</span>)</span><br><span class="line">password = <span class="string">'我是密碼-River'</span></span><br><span class="line">decrypted_xlsx_file_path = input_xlsx_file.dirname.join(<span class="string">'river_demo_decrypted.xlsx'</span>) <span class="comment"># 解密後的檔案</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 解密 xlsx 檔案</span></span><br><span class="line"><span class="string">`<span class="subst">#{msoffice}</span> -d -p <span class="subst">#{password}</span> <span class="subst">#{encrypted_xlsx_file_path}</span> <span class="subst">#{decrypted_xlsx_file_path}</span>`</span></span><br></pre></td></tr></table></figure></p><h3><span id="補充-隨機產密碼">補充: 隨機產密碼</span></h3><p>若密碼要改成隨機產生 (包含數字、英文大小寫) 的話，以下幾種方法皆可產密碼</p><p>隨機產的密碼，務必要存起來，不然加密後的 Excel 會不知道開啟的密碼 XD<br><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ openssl rand -base64 32</span><br><span class="line"></span><br><span class="line"><span class="comment"># macOS 需安裝 brew install pwgen</span></span><br><span class="line">$ pwgen 14 1 --symbols</span><br></pre></td></tr></table></figure></p><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># ruby</span></span><br><span class="line"><span class="comment"># need require 'securerandom'</span></span><br><span class="line"><span class="keyword">require</span> <span class="string">'securerandom'</span></span><br><span class="line"></span><br><span class="line">SecureRandom.hex</span><br><span class="line">SecureRandom.base64</span><br><span class="line">SecureRandom.alphanumeric(<span class="number">20</span>) <span class="comment"># 20 為密碼長度</span></span><br></pre></td></tr></table></figure></p><p><img src="ETt4MNY.png" alt></p><hr><h1><span id="幫-excel-欄位加超連結-修改字形色彩等">幫 Excel 欄位加超連結、修改字形色彩...等</span></h1><p>仍以上面 demo 資料為主(未加密的)<br>若需要幫已加上超連結的欄位資料，修改字形色彩、加粗體底線的話，這部分的範例，可參考此 <a href="https://github.com/River-Ye/ironman_13th_2021/pull/10" target="_blank" rel="noopener">pr</a></p><h2><span id="增加超連結-hyperlink">增加超連結 (hyperlink)</span></h2><p>commit 寫比較完整，可參考此 <a href="https://github.com/River-Ye/ironman_13th_2021/pull/10/commits/850fbddad22152f877e36bcbab3407459cfa5f26" target="_blank" rel="noopener">commit</a><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># app/services/shops_excel/generator.rb</span></span><br><span class="line"></span><br><span class="line">LINK_PATTERN = <span class="regexp">/\A\[(.*)\]\((http.*)\)\z/</span> <span class="comment"># 針對資料 "[data](https://riverye.com/shops/id)"，過濾成 data 和 https://riverye.com/shops/id</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">check_link</span><span class="params">(row_data)</span></span></span><br><span class="line"> row_data.each_with_index.map <span class="keyword">do</span> <span class="params">|data, index|</span></span><br><span class="line"> <span class="keyword">if</span> data.is_a?(String) && matched = data.match(LINK_PATTERN)</span><br><span class="line"> row_data[index] = matched[<span class="number">1</span>]</span><br><span class="line"> [index, matched[<span class="number">2</span>]]</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span>.compact</span><br><span class="line"><span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">add_link</span><span class="params">(sheet, links, cur_row)</span></span></span><br><span class="line"> links.map <span class="keyword">do</span> <span class="params">|index, link|</span></span><br><span class="line"> sheet.add_hyperlink(<span class="symbol">location:</span> link, <span class="symbol">ref:</span> sheet.rows[cur_row.row_index].cells[index])</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><h2><span id="增加篩選-凍結第一列">增加篩選、凍結第一列</span></h2><p>commit 寫比較完整，可參考此 <a href="https://github.com/River-Ye/ironman_13th_2021/pull/10/commits/1d0e45b6b665f22621689faca2c4edb5c80fb6eb" target="_blank" rel="noopener">commit</a><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># app/services/shops_excel/generator.rb</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 增加篩選</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">add_auto_filter</span><span class="params">(sheet)</span></span></span><br><span class="line"> number = TITLES.size - <span class="number">1</span></span><br><span class="line"> name = ((number % <span class="number">26</span>) + <span class="string">'A'</span>.ord).chr</span><br><span class="line"> excel_column_name =</span><br><span class="line"> <span class="keyword">if</span> number < <span class="number">26</span></span><br><span class="line"> name</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> (((number / <span class="number">26</span>) - <span class="number">1</span>) + <span class="string">'A'</span>.ord).chr + name</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> sheet.auto_filter = <span class="string">"A1:<span class="subst">#{excel_column_name}</span>1"</span></span><br><span class="line"><span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 凍結第一列</span></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">freeze_header</span><span class="params">(sheet)</span></span></span><br><span class="line"> sheet.sheet_view.pane <span class="keyword">do</span> <span class="params">|pane|</span></span><br><span class="line"> pane.top_left_cell = <span class="string">'B2'</span></span><br><span class="line"> pane.state = <span class="symbol">:frozen_split</span></span><br><span class="line"> pane.y_split = <span class="number">1</span></span><br><span class="line"> pane.x_split = <span class="number">0</span></span><br><span class="line"> pane.active_pane = <span class="symbol">:bottom_right</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><h2><span id="針對有超連結欄位改成藍色字形色彩-加底線">針對有超連結欄位，改成藍色字形色彩、加底線</span></h2><p>commit 寫比較完整，可參考此 <a href="https://github.com/River-Ye/ironman_13th_2021/pull/10/commits/ce1a12d4efbcfe7d18e368f6aae3bab8581accac" target="_blank" rel="noopener">commit</a><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># app/services/shops_excel/generator.rb</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">add_link</span><span class="params">(sheet, links, cur_row)</span></span></span><br><span class="line"> links.map <span class="keyword">do</span> <span class="params">|index, link|</span></span><br><span class="line"> ref = sheet.rows[cur_row.row_index].cells[index]</span><br><span class="line"> <span class="comment"># ref.r # A3</span></span><br><span class="line"> ref.color = <span class="string">'0000FF'</span> <span class="comment"># 藍色</span></span><br><span class="line"> ref.u = <span class="literal">true</span> <span class="comment"># 底線</span></span><br><span class="line"> sheet.add_hyperlink(<span class="symbol">location:</span> link, <span class="symbol">ref:</span> ref)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><h2><span id="補測試">補測試</span></h2><p>commit 寫比較完整，可參考此 <a href="https://github.com/River-Ye/ironman_13th_2021/pull/10/commits/50b9c1d9f72a90fd84d3eb1193072a055d66662b" target="_blank" rel="noopener">commit</a></p><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># spec/services/shops_excel/generator_spec.rb</span></span><br><span class="line"></span><br><span class="line"><span class="comment"># 驗證: 超連結</span></span><br><span class="line">expect_hyperlink = [first_sheet.hyperlinks.first.location, first_sheet.hyperlinks.first.ref]</span><br><span class="line">expect(expect_hyperlink).to match_array(</span><br><span class="line"> [<span class="string">"https://riverye.com/shops/<span class="subst">#{@shop.id}</span>"</span>, <span class="string">'A2'</span>]</span><br><span class="line">)</span><br><span class="line"><span class="comment"># 驗證: 顏色</span></span><br><span class="line">expect(first_sheet.rows[<span class="number">1</span>].cells.map(&<span class="symbol">:color</span>).map { <span class="params">|d|</span> d&.rgb }).to match_array(</span><br><span class="line"> [<span class="string">'FF0000FF'</span>, <span class="literal">nil</span>, <span class="literal">nil</span>]</span><br><span class="line">)</span><br><span class="line"><span class="comment"># 驗證: 底線</span></span><br><span class="line">expect(first_sheet.rows[<span class="number">1</span>].cells.map(&<span class="symbol">:u</span>)).to match_array(</span><br><span class="line"> [<span class="symbol">:single</span>, <span class="literal">nil</span>, <span class="literal">nil</span>]</span><br><span class="line">)</span><br></pre></td></tr></table></figure></p><h3><span id="實際結果">實際結果</span></h3><p><img src="NLdzsvQ.png" alt></p><h2><span id="小結">小結</span></h2><p>原本是只打算寫如何幫 Excel 的 xlsx 檔案加密，想說都寫了，順便查下如何解密，若有更好的方法，歡迎留言和我說，謝謝!!</p><p>寫加入超連結那段時，邊寫邊補資料 (原本就只是單純要寫如何加超連結，改字形色彩、下底線..等，是後來補的)，同時參考<a href="https://github.com/randym/axlsx/tree/master/examples" target="_blank" rel="noopener">官方範例</a>，網路搜尋下，也能查到許多可以參考的~</p><h2><span id="參考文件">參考文件</span></h2><ol><li><a href="https://github.com/roo-rb/roo/issues/399" target="_blank" rel="noopener">Opening password-protected excel files? #399</a></li><li><a href="https://github.com/randym/axlsx/tree/master/examples" target="_blank" rel="noopener">axlsx 官方文件</a></li></ol><hr><p>medium 文章連結：<a href="https://link.medium.com/EAV6xgl0fnb" target="_blank" rel="noopener">https://link.medium.com/EAV6xgl0fnb</a><br>本文同步發布於 <a href="https://riverye.com/2022/01/31/excel-advanced-skills/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>延續 <a href="https://riverye.com/2021/09/29/Day16-%E5%8C%AF%E5%87%BA-excel-%E6%87%89%E7%94%A8%E7%AF%87/">
</summary>
<category term="教學文" scheme="https://riverye.com/categories/%E6%95%99%E5%AD%B8%E6%96%87/"/>
<category term="Ruby on Rails" scheme="https://riverye.com/tags/Ruby-on-Rails/"/>
<category term="Excel" scheme="https://riverye.com/tags/Excel/"/>
<category term="xlsx" scheme="https://riverye.com/tags/xlsx/"/>
</entry>
<entry>
<title>用 Ruby 將訊息傳到 LINE 群組 (LINE Notify API)</title>
<link href="https://riverye.com/2022/01/30/line-notify/"/>
<id>https://riverye.com/2022/01/30/line-notify/</id>
<published>2022-01-29T16:00:00.000Z</published>
<updated>2022-12-21T14:28:51.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>首先你要先有 LINE，若沒有，可以左轉離開了 XD</p><h2><span id="步驟">步驟</span></h2><h3><span id="1-到-line-notify-並登入個人的-line-帳號">1. 到 LINE Notify ，並登入個人的 LINE 帳號</span></h3><p><a href="https://notify-bot.line.me/zh_TW/" target="_blank" rel="noopener">LINE Notify</a></p><h4><span id="line-notify-頁面">LINE Notify 頁面</span></h4><p><img src="jZ0yzC0.png" alt></p><h4><span id="登入自己的-line">登入自己的 LINE</span></h4><p><img src="Ppm0LQ2.png" alt></p><h3><span id="2-點自己的名字進入個人頁面">2. 點自己的名字，進入「個人頁面」</span></h3><p><img src="QP1t7Fi.png" alt></p><h3><span id="3-選發行權杖">3. 選「發行權杖」</span></h3><p><img src="4uuzfmL.png" alt></p><h3><span id="4-輸入想要加入-line-notify-的-line-群組接著點發行">4. 輸入想要加入 LINE Notify 的 LINE 群組，接著點「發行」</span></h3><p><img src="vu5wfPA.png" alt></p><h4><span id="注意-若離開此頁面將不會再顯示新發行的權杖-離開頁面前請先複製權杖">注意: 若離開此頁面，將不會再顯示新發行的權杖。 離開頁面前，請先複製權杖。</span></h4><p><img src="MKY6Vag.png" alt></p><h3><span id="5-用-curl-測試看看">5. 用 <code>curl</code> 測試看看</span></h3><h4><span id="記得先把line-notify-官方帳號加入群組中">記得先把「LINE Notify 官方帳號」加入群組中</span></h4><p><a href="https://liff.line.me/1645278921-kWRPP32q/?accountId=linenotify" target="_blank" rel="noopener">LINE Notify 官方帳號</a><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 記得換成自己的 token</span></span><br><span class="line">curl -X POST -H <span class="string">'Authorization: Bearer <access_token>'</span> -F <span class="string">'message=測試發訊息'</span> https://notify-api.line.me/api/notify</span><br><span class="line"></span><br><span class="line"><span class="comment"># 範例</span></span><br><span class="line">curl -X POST -H <span class="string">'Authorization: Bearer tsxxxxxxxxxIV'</span> -F <span class="string">'message=測試發訊息'</span> https://notify-api.line.me/api/notify</span><br></pre></td></tr></table></figure></p><p><a href="https://notify-bot.line.me/doc/en/#:~:text=Message%20visible%20to%20end%2Duser-,Sample,-%24%20curl%20%2DX%20POST%20%2DH%20%27Authorization%3A%20Bearer%20%3Caccess_token%3E%27%20%2DF" target="_blank" rel="noopener">官方文件 LINE Notify API Document 提供的範例</a></p><p><img src="ZrEaxSQ.png" alt></p><h3><span id="6-用-postman-測試看看">6. 用 Postman 測試看看</span></h3><p><img src="SZjbDxH.png" alt></p><p><img src="nsY9ALs.png" alt></p><h4><span id="postman-點-ltgt-icon-可以選各式程式語言的範例">Postman 點 <code></></code> icon ，可以選各式程式語言的範例</span></h4><p><img src="xF4Gnco.png" alt></p><h3><span id="7-用-ruby-的-irb-測試看看">7. 用 Ruby 的 <code>irb</code> 測試看看</span></h3><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">require</span>(<span class="string">'net/http'</span>)</span><br><span class="line">token = <span class="string">'tsxxxxxxxxxIV'</span> <span class="comment"># 換成自己的 Token</span></span><br><span class="line">url = <span class="string">'https://notify-api.line.me/api/notify'</span></span><br><span class="line">message = <span class="string">"測試要傳的訊息\nHello World!!"</span></span><br><span class="line">uri = URI(url)</span><br><span class="line">http = Net::HTTP.new(uri.host, uri.port)</span><br><span class="line">http.use_ssl = <span class="literal">true</span> <span class="keyword">if</span> uri.scheme == <span class="string">'https'</span></span><br><span class="line">http.open_timeout = <span class="number">10</span></span><br><span class="line">http.continue_timeout = <span class="number">10</span></span><br><span class="line"></span><br><span class="line">request = Net::HTTP::Post.new(uri)</span><br><span class="line">request[<span class="string">'Authorization'</span>] = <span class="string">"Bearer <span class="subst">#{token}</span>"</span></span><br><span class="line">request.set_form_data(<span class="symbol">message:</span> <span class="string">"\n<span class="subst">#{message}</span>"</span>)</span><br><span class="line">response = http.request(request)</span><br><span class="line">response</span><br></pre></td></tr></table></figure></p><p><img src="f2KCVif.png" alt></p><h2><span id="將-ruby-那段寫得更嚴謹些">將 Ruby 那段寫得更嚴謹些</span></h2><p>不能保證每次打 API 都是正常的，有可能剛好遇到 LINE 服務異常，導致 <a href="https://github.com/ankane/the-ultimate-guide-to-ruby-timeouts" target="_blank" rel="noopener">Timeout</a> 等問題，應該要針對已知可能會發生的問題，進行預防和處理。</p><p>另外也應該針對 response 和 exception 存 logger，後續有異常要追問題時，才會有比較多線索和方向</p><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">ChatBotNotifier</span></span></span><br><span class="line"> <span class="keyword">require</span>(<span class="string">'logger'</span>)</span><br><span class="line"> <span class="keyword">require</span>(<span class="string">'net/http'</span>)</span><br><span class="line"></span><br><span class="line"> LINE_NOTIFY_URL = <span class="string">'https://notify-api.line.me/api/notify'</span>.freeze</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">line_notify</span><span class="params">(token, message)</span></span></span><br><span class="line"> uri = URI(LINE_NOTIFY_URL)</span><br><span class="line"> http = init_http(uri)</span><br><span class="line"> call_line_api(uri, http, token, message)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> private</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">logger</span></span></span><br><span class="line"> @logger <span class="params">||</span>= Logger.new(<span class="string">'message.log'</span>)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">init_http</span><span class="params">(uri)</span></span></span><br><span class="line"> http = Net::HTTP.new(uri.host, uri.port)</span><br><span class="line"> http.use_ssl = <span class="literal">true</span> <span class="keyword">if</span> uri.scheme == <span class="string">'https'</span></span><br><span class="line"> http.open_timeout = <span class="number">10</span></span><br><span class="line"> http.continue_timeout = <span class="number">10</span></span><br><span class="line"> http</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">call_line_api</span><span class="params">(uri, http, token, message, error_retry = <span class="number">0</span>)</span></span></span><br><span class="line"> request = Net::HTTP::Post.new(uri)</span><br><span class="line"> request[<span class="string">'Authorization'</span>] = <span class="string">"Bearer <span class="subst">#{token}</span>"</span></span><br><span class="line"> request.set_form_data(<span class="symbol">message:</span> <span class="string">"\n<span class="subst">#{message}</span>"</span>)</span><br><span class="line"> response = http.request(request)</span><br><span class="line"> logger.info(<span class="string">"<span class="subst">#{__FILE_<span class="number">_</span>}</span>#<span class="subst">#{__method_<span class="number">_</span>}</span> response: <span class="subst">#{response.body}</span>"</span>)</span><br><span class="line"> <span class="keyword">rescue</span> StandardError => e</span><br><span class="line"> <span class="keyword">if</span> error_retry < <span class="number">5</span></span><br><span class="line"> sleep(<span class="number">10</span>)</span><br><span class="line"> error_retry += <span class="number">1</span></span><br><span class="line"> <span class="keyword">retry</span></span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> logger.error(<span class="string">"<span class="subst">#{__FILE_<span class="number">_</span>}</span>#<span class="subst">#{__method_<span class="number">_</span>}</span> exception: <span class="subst">#{e.inspect}</span>"</span>)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"></span><br><span class="line">token = <span class="string">'tsxxxxxxxxxIV'</span> <span class="comment"># 換成自己的 Token</span></span><br><span class="line">message = <span class="string">"測試要傳的訊息\nHello World!!"</span></span><br><span class="line">ChatBotNotifier.new.line_notify(token, message)</span><br></pre></td></tr></table></figure></p><h2><span id="小結">小結</span></h2><p>若有例行要傳的訊息，可以用 Linux crontab 做排程，也可用 Sidekiq 的 scheduler 處理。</p><h2><span id="參考文件">參考文件</span></h2><p><a href="https://notify-bot.line.me/doc/en/" target="_blank" rel="noopener">LINE Notify API Document</a></p><hr><p>medium 文章連結：<a href="https://link.medium.com/gkQQBO1Henb" target="_blank" rel="noopener">https://link.medium.com/gkQQBO1Henb</a><br>本文同步發布於 <a href="https://riverye.com/2022/01/30/line-notify/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>首先你要先有 LINE，若沒有，可以左轉離開了 XD</p>
<h2><span id="步驟">步驟</span></h2>
<h3><span id="1-到-line-notify-並登入個人的-lin
</summary>
<category term="教學文" scheme="https://riverye.com/categories/%E6%95%99%E5%AD%B8%E6%96%87/"/>
<category term="Postman" scheme="https://riverye.com/tags/Postman/"/>
<category term="Ruby" scheme="https://riverye.com/tags/Ruby/"/>
<category term="curl" scheme="https://riverye.com/tags/curl/"/>
<category term="LINE Notify" scheme="https://riverye.com/tags/LINE-Notify/"/>
</entry>
<entry>
<title>Day30 - 終於完賽的心得感言</title>
<link href="https://riverye.com/2021/10/13/Day30-%E7%B5%82%E6%96%BC%E5%AE%8C%E8%B3%BD%E7%9A%84%E5%BF%83%E5%BE%97%E6%84%9F%E8%A8%80/"/>
<id>https://riverye.com/2021/10/13/Day30-終於完賽的心得感言/</id>
<published>2021-10-12T16:00:00.000Z</published>
<updated>2022-09-23T08:45:10.000Z</updated>
<content type="html"><![CDATA[<h2><span id="參賽動機">參賽動機</span></h2><p>老實說，今年其實沒有打算要參賽，打算裝死度過</p><p>剛好身邊有朋友要轉職，鼓勵對方說可以參加鐵人賽，沒想到對方答應了...</p><p>只好硬著頭皮跟著參賽，總不能只在旁邊喊燒吧</p><p>雖然報名的前 2 個月就已經知道躲不了</p><p>一開始還會和對方說，我們就先每週寫 1-2 篇，然後傳給對方，互相鼓勵 <s>互相傷害</s></p><p>想像中的劇本是 2 個月前開始慢慢準備</p><p>到開賽前便累積 30 篇文章，接著只要每天無壓力 po 文即可</p><p>But，代誌不是憨人想得那麼簡單</p><p>殊不知自己惰性出現，事前只準備了 12 篇左右的文章...</p><p>其他 18 篇左右則是在報名後開始寫的</p><p>邊工作同時還要寫鐵人賽文章，真的蠻硬的</p><p>無數個夜晚、週末都在寫鐵人賽文章</p><p>好在有一些的庫存，壓力不會像去年一樣這麼大</p><p>建議要參加的人，可以先準備庫存</p><p>都不準備直接每天寫的話，會非常的嗨 (過來人經驗)</p><h2><span id="總結">總結</span></h2><p>事前就開始思考整體的架構，果不其然，預期與實際有些出入 (笑)</p><p>整體應該算還可以吧 (?)</p><p>前半段左右的文章，蠻多都是工作中累積的經驗</p><p>無意間知道有些同事知道我在寫鐵人賽文章</p><p>甚至連其他部門的人也知道...</p><p>挺害羞的，想說低調寫低調完賽就好</p><p>這邊要聲明下</p><p>後半段寫與「股票」相關的文章，是我在 3-4 個月前下班與週末心血來潮寫的 Side Project</p><p>絕不是上班時寫的，才不是薪水小偷呢!! <s>你才薪水小偷，你全家都薪水小偷</s></p><p>另外，送上其中一首好聽的音樂與大家分享 <a href="https://youtu.be/sHxh3wvnE6g" target="_blank" rel="noopener">與你更靠近(Closer To You)</a></p><p>沒意外的話，明年不會參加了，連續參加三屆的鐵人賽，應該夠了吧(逃) xDD</p><p>最後，「你必須很努力，才能看起來毫不費力」</p><p>謝謝大家的觀看</p><h2><span id="大補帖">大補帖</span></h2><table><thead><tr><th style="text-align:center">編號</th><th>名稱</th><th style="text-align:center">小菜的 Blog</th><th style="text-align:center">鐵人賽</th><th style="text-align:center">Medium</th></tr></thead><tbody><tr><td style="text-align:center">01</td><td>Day01 - 鐵人賽我又來囉</td><td style="text-align:center"><a href="http://riverye.com/2021/09/14/Day01-%E9%90%B5%E4%BA%BA%E8%B3%BD%E6%88%91%E5%8F%88%E4%BE%86%E5%9B%89/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10264068" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/rhBstgf2Mjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">02</td><td>Day02 - 修改 Rails console edit 編輯模式</td><td style="text-align:center"><a href="http://riverye.com/2021/09/15/Day02-%E4%BF%AE%E6%94%B9-Rails-console-edit-%E7%B7%A8%E8%BC%AF%E6%A8%A1%E5%BC%8F">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10264100" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/L7ujeSg2Mjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">03</td><td>Day03 - Gem-strip_attributes 介紹與應用</td><td style="text-align:center"><a href="http://riverye.com/2021/09/16/Day03-Gem-strip-attributes-%E4%BB%8B%E7%B4%B9%E8%88%87%E6%87%89%E7%94%A8">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10264570" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/tBNoHEh2Mjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">04</td><td>Day04 - Gem-activerecord-import 批次建立介紹與應用</td><td style="text-align:center"><a href="http://riverye.com/2021/09/17/Day04-Gem-activerecord-import-%E6%89%B9%E6%AC%A1%E5%BB%BA%E7%AB%8B%E4%BB%8B%E7%B4%B9%E8%88%87%E6%87%89%E7%94%A8/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10264572" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/GX2nvti2Mjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">05</td><td>Day05 - Gem-paranoia 軟刪除介紹與應用</td><td style="text-align:center"><a href="http://riverye.com/2021/09/18/Day05-Gem-paranoia-%E8%BB%9F%E5%88%AA%E9%99%A4%E4%BB%8B%E7%B4%B9%E8%88%87%E6%87%89%E7%94%A8/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10264573" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/ay1JSdj2Mjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">06</td><td>Day06 - 監控 Sidekiq 有無正常運作(或執行超過多久)</td><td style="text-align:center"><a href="http://riverye.com/2021/09/19/Day06-%E7%9B%A3%E6%8E%A7-Sidekiq-%E6%9C%89%E7%84%A1%E6%AD%A3%E5%B8%B8%E9%81%8B%E4%BD%9C-%E6%88%96%E5%9F%B7%E8%A1%8C%E8%B6%85%E9%81%8E%E5%A4%9A%E4%B9%85/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10264574" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/mtoj9Ck2Mjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">07</td><td>Day07 - Gem-sidekiq-limit_fetch 限制 sidekiq queue 執行數量</td><td style="text-align:center"><a href="https://riverye.com/2021/09/20/Day07-Gem-sidekiq-limit-fetch-%E9%99%90%E5%88%B6-sidekiq-queue-%E5%9F%B7%E8%A1%8C%E6%95%B8%E9%87%8F/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10264576" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/WH74csl2Mjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">08</td><td>Day08 - Gem-sidekiq-grouping 允許單個 sidekiq 處理多個相似(一樣)的 jobs</td><td style="text-align:center"><a href="http://riverye.com/2021/09/21/Day08-Gem-sidekiq-grouping-%E5%85%81%E8%A8%B1%E5%96%AE%E5%80%8B-sidekiq-%E8%99%95%E7%90%86%E5%A4%9A%E5%80%8B%E7%9B%B8%E4%BC%BC-%E4%B8%80%E6%A8%A3-%E7%9A%84-jobs">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10264578" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/0SSdkIm2Mjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">09</td><td>Day09 - Gem-jwt 介紹與應用</td><td style="text-align:center"><a href="http://riverye.com/2021/09/22/Day09-Gem-jwt-%E4%BB%8B%E7%B4%B9%E8%88%87%E6%87%89%E7%94%A8/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10264580" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/OYkJftn2Mjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">10</td><td>Day10 - 如何用手機連 Ruby on Rails Local 開發中的專案</td><td style="text-align:center"><a href="http://riverye.com/2021/09/23/Day10-%E5%A6%82%E4%BD%95%E7%94%A8%E6%89%8B%E6%A9%9F%E9%80%A3-Ruby-on-Rails-Local-%E9%96%8B%E7%99%BC%E4%B8%AD%E7%9A%84%E5%B0%88%E6%A1%88/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10264598" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/wiiTrho2Mjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">11</td><td>Day11 - Gem-rqrcode 或 barby 產 QR Code</td><td style="text-align:center"><a href="http://riverye.com/2021/09/24/Day11-Gem-rqrcode-%E6%88%96-barby-%E7%94%A2-QR-Code/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10264600" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/ray451o2Mjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">12</td><td>Day12 - 解析圖片中的 QR Code 資料</td><td style="text-align:center"><a href="http://riverye.com/2021/09/25/Day12-%E8%A7%A3%E6%9E%90%E5%9C%96%E7%89%87%E4%B8%AD%E7%9A%84-QR-Code-%E8%B3%87%E6%96%99/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10264602" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/MDSrOdpGOjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">13</td><td>Day13 - PDF 加密、解密的處理</td><td style="text-align:center"><a href="http://riverye.com/2021/09/26/Day13-PDF-%E5%8A%A0%E5%AF%86%E3%80%81%E8%A7%A3%E5%AF%86%E7%9A%84%E8%99%95%E7%90%86/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10271819" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/JoYN5HDVPjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">14</td><td>Day14 - PDF 加浮水印</td><td style="text-align:center"><a href="http://riverye.com/2021/09/27/Day14-PDF-%E5%8A%A0%E6%B5%AE%E6%B0%B4%E5%8D%B0/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10271878" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/uqXuO7EVPjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">15</td><td>Day15 - 匯出(下載) PDF</td><td style="text-align:center"><a href="http://riverye.com/2021/09/28/Day15-%E5%8C%AF%E5%87%BA-%E4%B8%8B%E8%BC%89-PDF/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10271932" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/91hPYYFVPjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">16</td><td>Day16 - 匯出 excel-應用篇</td><td style="text-align:center"><a href="http://riverye.com/2021/09/29/Day16-%E5%8C%AF%E5%87%BA-excel-%E6%87%89%E7%94%A8%E7%AF%87/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10272151" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/GmNaDhwgQjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">17</td><td>Day17 - 匯出 excel-測試篇</td><td style="text-align:center"><a href="http://riverye.com/2021/09/30/Day17-%E5%8C%AF%E5%87%BA-excel-%E6%B8%AC%E8%A9%A6%E7%AF%87/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10272496" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/H7KaCZbhRjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">18</td><td>Day18 - 匯入 excel-應用篇</td><td style="text-align:center"><a href="http://riverye.com/2021/10/01/Day18-%E5%8C%AF%E5%85%A5-excel-%E6%87%89%E7%94%A8%E7%AF%87/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10272591" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/Teq2MzayRjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">19</td><td>Day19 - 匯入 excel-測試篇</td><td style="text-align:center"><a href="http://riverye.com/2021/10/02/Day19-%E5%8C%AF%E5%85%A5-excel-%E6%B8%AC%E8%A9%A6%E7%AF%87/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10272599" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/sYWyCHbyRjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">20</td><td>Day20 - 用 Ruby on Rails 抓臺灣證券交易所資料-每日收盤行情</td><td style="text-align:center"><a href="http://riverye.com/2021/10/03/Day20-%E7%94%A8-Ruby-on-Rails-%E6%8A%93%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E8%B3%87%E6%96%99-%E6%AF%8F%E6%97%A5%E6%94%B6%E7%9B%A4%E8%A1%8C%E6%83%85/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10272778" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/DtL2qRIuTjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">21</td><td>Day21 - 用 Ruby on Rails 抓臺灣證券交易所資料-除權除息計算結果表</td><td style="text-align:center"><a href="http://riverye.com/2021/10/04/Day21-%E7%94%A8-Ruby-on-Rails-%E6%8A%93%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E8%B3%87%E6%96%99-%E9%99%A4%E6%AC%8A%E9%99%A4%E6%81%AF%E8%A8%88%E7%AE%97%E7%B5%90%E6%9E%9C%E8%A1%A8/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10272820" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/FasXk7JuTjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">22</td><td>Day22 - 用 Ruby on Rails 處理臺灣證券交易所資料-DB 設計</td><td style="text-align:center"><a href="http://riverye.com/2021/10/05/Day22-%E7%94%A8-Ruby-on-Rails-%E8%99%95%E7%90%86%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E8%B3%87%E6%96%99-DB-%E8%A8%AD%E8%A8%88/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10272854" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/s8TWR4KuTjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">23</td><td>Day23 - 將臺灣證券交易所的每日收盤行情存入 DB</td><td style="text-align:center"><a href="http://riverye.com/2021/10/06/Day23-%E5%B0%87%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E7%9A%84%E6%AF%8F%E6%97%A5%E6%94%B6%E7%9B%A4%E8%A1%8C%E6%83%85%E5%AD%98%E5%85%A5-DB/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10272899" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/HmmocOLuTjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">24</td><td>Day24 - 將臺灣證券交易所的除權除息計算結果表存入 DB</td><td style="text-align:center"><a href="http://riverye.com/2021/10/07/Day24-%E5%B0%87%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E7%9A%84%E9%99%A4%E6%AC%8A%E9%99%A4%E6%81%AF%E8%A8%88%E7%AE%97%E7%B5%90%E6%9E%9C%E8%A1%A8%E5%AD%98%E5%85%A5-DB/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10272913" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/VfJfmvMuTjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">25</td><td>Day25 - 透過 Rake 自動下載處理臺灣證券交易所的資料</td><td style="text-align:center"><a href="http://riverye.com/2021/10/08/Day25-%E9%80%8F%E9%81%8E-Rake-%E8%87%AA%E5%8B%95%E4%B8%8B%E8%BC%89%E8%99%95%E7%90%86%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E7%9A%84%E8%B3%87%E6%96%99/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10273391" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/wzKY4dNuTjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">26</td><td>Day26 - 用 Ruby on Rails 寫分析股票的技術指標</td><td style="text-align:center"><a href="http://riverye.com/2021/10/09/Day26-%E7%94%A8-Ruby-on-Rails-%E5%AF%AB%E5%88%86%E6%9E%90%E8%82%A1%E7%A5%A8%E7%9A%84%E6%8A%80%E8%A1%93%E6%8C%87%E6%A8%99/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10273425" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/G7zXw5NuTjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">27</td><td>Day27 - 如何讓 Google 搜尋到你的網站</td><td style="text-align:center"><a href="http://riverye.com/2021/10/10/Day27-%E5%A6%82%E4%BD%95%E8%AE%93-Google-%E6%90%9C%E5%B0%8B%E5%88%B0%E4%BD%A0%E7%9A%84%E7%B6%B2%E7%AB%99/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10274871" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/LgXnsCqVWjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">28</td><td>Day28 - 如何埋 GA (Google Analytics)</td><td style="text-align:center"><a href="http://riverye.com/2021/10/11/Day28-%E5%A6%82%E4%BD%95%E5%9F%8B-GA-Google-Analytics/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10275551" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/c5HrqkbCYjb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">29</td><td>Day29 - 透過 PageSpeed Insights 了解網站速度優化</td><td style="text-align:center"><a href="http://riverye.com/2021/10/12/Day29-%E9%80%8F%E9%81%8E-PageSpeed-Insights-%E4%BA%86%E8%A7%A3%E7%B6%B2%E7%AB%99%E9%80%9F%E5%BA%A6%E5%84%AA%E5%8C%96/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10276008" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/CVmaqyOb0jb" target="_blank" rel="noopener">點我</a></td></tr><tr><td style="text-align:center">30</td><td>Day30 - 終於完賽的心得感言</td><td style="text-align:center"><a href="http://riverye.com/2021/10/13/Day30-%E7%B5%82%E6%96%BC%E5%AE%8C%E8%B3%BD%E7%9A%84%E5%BF%83%E5%BE%97%E6%84%9F%E8%A8%80/">點我</a></td><td style="text-align:center"><a href="https://ithelp.ithome.com.tw/articles/10276149" target="_blank" rel="noopener">點我</a></td><td style="text-align:center"><a href="https://link.medium.com/XwwhFnnm0jb" target="_blank" rel="noopener">點我</a></td></tr></tbody></table><hr><p>鐵人賽文章連結：<a href="https://ithelp.ithome.com.tw/articles/10276149" target="_blank" rel="noopener">https://ithelp.ithome.com.tw/articles/10276149</a><br>medium 文章連結：<a href="https://link.medium.com/XwwhFnnm0jb" target="_blank" rel="noopener">https://link.medium.com/XwwhFnnm0jb</a><br>本文同步發布於 <a href="http://riverye.com/2021/10/13/Day30-%E7%B5%82%E6%96%BC%E5%AE%8C%E8%B3%BD%E7%9A%84%E5%BF%83%E5%BE%97%E6%84%9F%E8%A8%80/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="參賽動機">參賽動機</span></h2>
<p>老實說，今年其實沒有打算要參賽，打算裝死度過</p>
<p>剛好身邊有朋友要轉職，鼓勵對方說可以參加鐵人賽，沒想到對方答應了...</p>
<p>只好硬著頭皮跟著參賽，總不能只在旁邊喊燒吧</p>
<
</summary>
<category term="鐵人賽" scheme="https://riverye.com/categories/%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
<category term="13th鐵人賽" scheme="https://riverye.com/tags/13th%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
</entry>
<entry>
<title>Day29 - 透過 PageSpeed Insights 了解網站速度優化</title>
<link href="https://riverye.com/2021/10/12/Day29-%E9%80%8F%E9%81%8E-PageSpeed-Insights-%E4%BA%86%E8%A7%A3%E7%B6%B2%E7%AB%99%E9%80%9F%E5%BA%A6%E5%84%AA%E5%8C%96/"/>
<id>https://riverye.com/2021/10/12/Day29-透過-PageSpeed-Insights-了解網站速度優化/</id>
<published>2021-10-11T16:00:00.000Z</published>
<updated>2022-12-21T14:32:44.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>若想進行網站速度優化，Google 有提供 <a href="https://developers.google.com/speed/pagespeed/insights/" target="_blank" rel="noopener">PageSpeed Insights</a> 與 <a href="https://www.thinkwithgoogle.com/intl/zh-tw/feature/testmysite/" target="_blank" rel="noopener">Think with Google</a> 這兩項工具可供參考</p><h2><span id="說明">說明</span></h2><p>使用方式非常的簡單，將想要測試的網址，輸入在 Google 提供的網站中進行測試即可</p><h3><span id="pagespeed-insights">PageSpeed Insights</span></h3><p><a href="https://developers.google.com/speed/pagespeed/insights/" target="_blank" rel="noopener">https://developers.google.com/speed/pagespeed/insights/</a></p><p><img src="3LNUsA5.png" alt></p><h4><span id="試著把-google-提供的網址輸入進去看看">試著把 Google 提供的網址輸入進去看看...</span></h4><p>以下為行動版的分數</p><p><img src="uaD8fVb.png" alt></p><p>以下為電腦版的分數</p><p><img src="2ZQfTri.png" alt></p><h4><span id="實際跑完顯示的詳細資訊">實際跑完顯示的詳細資訊</span></h4><p>發現同樣的網址，連續測試時，每次的分數不完全一樣，當參考即可</p><p><img src="HlacHUr.jpg" alt></p><hr><h3><span id="比較您的行動網站載入速度-think-with-google">比較您的行動網站載入速度 - Think with Google</span></h3><p><a href="https://www.thinkwithgoogle.com/intl/zh-tw/feature/testmysite/" target="_blank" rel="noopener">https://www.thinkwithgoogle.com/intl/zh-tw/feature/testmysite/</a></p><p><img src="ufCOhtV.png" alt></p><h4><span id="試著把-google-提供的網址輸入進去看看">試著把 Google 提供的網址輸入進去看看...</span></h4><p><img src="eIGeHNJ.png" alt></p><h4><span id="實際跑完顯示的詳細資訊">實際跑完顯示的詳細資訊</span></h4><p>頁面的左上角可以「取得完整報表」</p><p><img src="PEoqcEY.jpg" alt></p><h2><span id="小結">小結</span></h2><p>跑出來的結果，可以當作參考，在自行評估是否要調整</p><p>也可以看下自己公司、競爭對手的網站的分數如何，分數只是分數，不能代表什麼，當作茶餘飯後閒聊即可</p><hr><p>鐵人賽文章連結：<a href="https://ithelp.ithome.com.tw/articles/10276008" target="_blank" rel="noopener">https://ithelp.ithome.com.tw/articles/10276008</a><br>medium 文章連結：<a href="https://link.medium.com/CVmaqyOb0jb" target="_blank" rel="noopener">https://link.medium.com/CVmaqyOb0jb</a><br>本文同步發布於 <a href="http://riverye.com/2021/10/12/Day29-%E9%80%8F%E9%81%8E-PageSpeed-Insights-%E4%BA%86%E8%A7%A3%E7%B6%B2%E7%AB%99%E9%80%9F%E5%BA%A6%E5%84%AA%E5%8C%96/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>若想進行網站速度優化，Google 有提供 <a href="https://developers.google.com/speed/pagespeed/insights/" target="_blank"
</summary>
<category term="鐵人賽" scheme="https://riverye.com/categories/%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
<category term="13th鐵人賽" scheme="https://riverye.com/tags/13th%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
</entry>
<entry>
<title>Day28 - 如何埋 GA (Google Analytics)</title>
<link href="https://riverye.com/2021/10/11/Day28-%E5%A6%82%E4%BD%95%E5%9F%8B-GA-Google-Analytics/"/>
<id>https://riverye.com/2021/10/11/Day28-如何埋-GA-Google-Analytics/</id>
<published>2021-10-10T16:00:00.000Z</published>
<updated>2022-12-21T14:30:32.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p><a href="https://ithelp.ithome.com.tw/articles/10274871" target="_blank" rel="noopener">前篇教學</a>已經能讓 Google 搜尋到網站，接下來若想做流量分析的話，需要埋 <a href="https://analytics.google.com/analytics/" target="_blank" rel="noopener">Google Analytics</a> (簡稱: GA)</p><h2><span id="如何使用">如何使用</span></h2><p>首先要有 Google 帳號，接著到這裡<a href="https://analytics.google.com/analytics/web/?authuser=0#provision/SignUp/" target="_blank" rel="noopener">註冊 Google Analytics</a>，照著網頁提示註冊即可</p><p><img src="sMLU5ab.png" alt></p><h3><span id="註冊完-ga-後點選網站">註冊完 GA 後，點選「網站」</span></h3><p><img src="ldU3Eyz.png" alt></p><h3><span id="網站網址-串流名稱要填寫">網站網址、串流名稱，要填寫</span></h3><p><img src="fhDTNYX.png" alt></p><h3><span id="這邊以全域網站代碼為例">這邊以「全域網站代碼」為例</span></h3><p>Google 會提供一組 HTML HEAD，貼到自己專案中的 HTML HEAD 中，並部署</p><p><img src="5rX52XW.png" alt></p><h3><span id="檢查自己專案是否有埋成功">檢查自己專案是否有埋成功</span></h3><p>可透過開發者工具 (快捷鍵 F12)，並搜尋 <code>GoogleAnalyticsObject</code> ，看是否有找到對應的 code</p><p><img src="2trPhLC.jpg" alt></p><h3><span id="回到-ga-檢查是否有流量">回到 GA 檢查是否有流量</span></h3><p>上個步驟檢查有的話，理論上就大功告成了</p><p><img src="YfPhJQ1.png" alt></p><h2><span id="小結">小結</span></h2><p>對行銷人來說，這是非常基本的事情，但對於初次接觸且不熟行銷的人來說，會需要稍微了解下</p><hr><p>鐵人賽文章連結：<a href="https://ithelp.ithome.com.tw/articles/10275551" target="_blank" rel="noopener">https://ithelp.ithome.com.tw/articles/10275551</a><br>medium 文章連結：<a href="https://link.medium.com/c5HrqkbCYjb" target="_blank" rel="noopener">https://link.medium.com/c5HrqkbCYjb</a><br>本文同步發布於 <a href="http://riverye.com/2021/10/11/Day28-%E5%A6%82%E4%BD%95%E5%9F%8B-GA-Google-Analytics/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p><a href="https://ithelp.ithome.com.tw/articles/10274871" target="_blank" rel="noopener">前篇教學</a>已經能讓 Goo
</summary>
<category term="鐵人賽" scheme="https://riverye.com/categories/%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
<category term="13th鐵人賽" scheme="https://riverye.com/tags/13th%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
</entry>
<entry>
<title>Day27 - 如何讓 Google 搜尋到你的網站</title>
<link href="https://riverye.com/2021/10/10/Day27-%E5%A6%82%E4%BD%95%E8%AE%93-Google-%E6%90%9C%E5%B0%8B%E5%88%B0%E4%BD%A0%E7%9A%84%E7%B6%B2%E7%AB%99/"/>
<id>https://riverye.com/2021/10/10/Day27-如何讓-Google-搜尋到你的網站/</id>
<published>2021-10-09T16:00:00.000Z</published>
<updated>2022-12-21T14:34:26.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>網站上線後，希望給更多人能找到的話，通常會用 <a href="https://search.google.com/search-console/about" target="_blank" rel="noopener">Google Search Console</a>，讓自己的網站可以被 Google 搜尋的到</p><p>note: 已經會設定的人，可以左轉離開了~</p><h2><span id="如何使用">如何使用</span></h2><p>到 <a href="https://search.google.com/search-console/about" target="_blank" rel="noopener">https://search.google.com/search-console/about</a> ，Google 很貼心提供中文介面，基本上就是照著步驟做，網站下方也有提供教學影片</p><p><img src="RpyQpgS.png" alt></p><h3><span id="可以選擇輸入網域或網址前置字元以下兩種驗證方式會不同會各別示範">可以選擇輸入「網域」或「網址前置字元」，以下兩種驗證方式會不同，會各別示範</span></h3><p><img src="LHMz0BC.png" alt></p><h3><span id="網址前置字元提供以下幾種驗證">「網址前置字元」提供以下幾種驗證</span></h3><ol><li>HTML 檔案 (建議用這個)</li><li>HTML 標記</li><li>Google Analytics (分析)</li><li>Google 代碼管理工具</li><li>網域名稱供應商</li></ol><p>Google 會提供 HTML 檔案，跟著放到專案(部落格)中，一起部署後，就能在回到這頁做驗證了</p><p>note: 驗證成功後，也不能移除喔~</p><p><img src="tYV9oet.png" alt></p><h3><span id="網域驗證">「網域」驗證</span></h3><p>我自己是在 <a href="https://www.gandi.net/" target="_blank" rel="noopener">Gandi.net</a> 買網域的</p><p>直接輸入網域後，會跳出如下視窗，接著進行第三方驗證即可</p><p><img src="7UKU3rc.png" alt></p><h3><span id="網域驗證完成畫面如下">「網域」驗證完成畫面如下</span></h3><p><img src="bovQvOo.png" alt></p><h3><span id="接著等待-google-資料處理約一天時間通常會更快">接著等待 Google 資料處理，約一天時間(通常會更快)</span></h3><p>一天後再來看，就能知道是否設定成功了<br><img src="KXiW8qD.png" alt></p><h3><span id="可用之後的畫面">可用之後的畫面</span></h3><p><img src="Gp3tfz4.png" alt></p><h2><span id="小結">小結</span></h2><p>這篇是定位在完全不知道如何讓自己網站被 Google 搜尋到的小白看的，被搜尋到只是剛起步，後續如何導流量、SEO、使用者體驗...等，可在自行研究或請益身邊的行銷好友們~</p><hr><p>鐵人賽文章連結：<a href="https://ithelp.ithome.com.tw/articles/10274871" target="_blank" rel="noopener">https://ithelp.ithome.com.tw/articles/10274871</a><br>medium 文章連結：<a href="https://link.medium.com/LgXnsCqVWjb" target="_blank" rel="noopener">https://link.medium.com/LgXnsCqVWjb</a><br>本文同步發布於 <a href="http://riverye.com/2021/10/10/Day27-%E5%A6%82%E4%BD%95%E8%AE%93-Google-%E6%90%9C%E5%B0%8B%E5%88%B0%E4%BD%A0%E7%9A%84%E7%B6%B2%E7%AB%99/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>網站上線後，希望給更多人能找到的話，通常會用 <a href="https://search.google.com/search-console/about" target="_blank" rel="noo
</summary>
<category term="鐵人賽" scheme="https://riverye.com/categories/%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
<category term="13th鐵人賽" scheme="https://riverye.com/tags/13th%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
</entry>
<entry>
<title>Day26 - 用 Ruby on Rails 寫分析股票的技術指標</title>
<link href="https://riverye.com/2021/10/09/Day26-%E7%94%A8-Ruby-on-Rails-%E5%AF%AB%E5%88%86%E6%9E%90%E8%82%A1%E7%A5%A8%E7%9A%84%E6%8A%80%E8%A1%93%E6%8C%87%E6%A8%99/"/>
<id>https://riverye.com/2021/10/09/Day26-用-Ruby-on-Rails-寫分析股票的技術指標/</id>
<published>2021-10-08T16:00:00.000Z</published>
<updated>2022-12-21T14:30:06.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>在做選股之前，可透過技術指標來分析，像是可以用 KD、均線、量價...等</p><h2><span id="說明">說明</span></h2><p>這邊要申明下，這邊是以技術交流為主，不會有任何投資建議，當自己寫了一套選股策略後，除了要做回測外，要知道沒有 100% 勝率且永遠有效的方法，如果有，請私訊讓我知道 xDD</p><h2><span id="實作">實作</span></h2><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># app/models/stock.rb</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Stock</span> < ApplicationRecord</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">specify_date</span><span class="params">(date)</span></span></span><br><span class="line"> daily_quotes.find_by(<span class="symbol">transaction_date:</span> date.to_date)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">self</span>.<span class="title">simple_moving_average</span><span class="params">(code, number)</span></span></span><br><span class="line"> stock = <span class="keyword">self</span>.find_by!(<span class="symbol">code:</span> code)</span><br><span class="line"> stock.simple_moving_average(number)</span><br><span class="line"> <span class="keyword">rescue</span> ActiveRecord::RecordNotFound => e</span><br><span class="line"> puts <span class="string">"無此股票代號: <span class="subst">#{code}</span>"</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">simple_moving_average</span><span class="params">(number)</span></span></span><br><span class="line"> result = daily_quotes.order(<span class="string">"transaction_date DESC"</span>).limit(number).select(<span class="symbol">:closing_price</span>)</span><br><span class="line"> <span class="keyword">if</span> result.size == number</span><br><span class="line"> (result.sum(&<span class="symbol">:closing_price</span>) / number).round(<span class="number">2</span>)</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> <span class="number">0</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 找出指定日期，成交量前幾名的股票</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">self</span>.<span class="title">top_ranks_trade_volume</span><span class="params">(number, date = <span class="literal">nil</span>)</span></span></span><br><span class="line"> <span class="keyword">self</span>.latest_transaction_date(date)</span><br><span class="line"> .order(<span class="string">"daily_quotes.trade_volume DESC"</span>)</span><br><span class="line"> .limit(number)</span><br><span class="line"> .select(<span class="symbol">:name</span>, <span class="symbol">:code</span>, <span class="string">"daily_quotes.trade_volume"</span>)</span><br><span class="line"> .map { <span class="params">|stock|</span> { <span class="string">"股票名稱"</span> => stock.name, <span class="string">"股票代號"</span> => stock.code, <span class="string">"成交量(張)"</span> => stock.trade_volume / <span class="number">1000</span> } }</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 找出指定日期，成交量破XXXX張的股票</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">self</span>.<span class="title">volume_broke_lot</span><span class="params">(number, date = <span class="literal">nil</span>)</span></span></span><br><span class="line"> <span class="keyword">self</span>.latest_transaction_date(date)</span><br><span class="line"> .where(<span class="string">"daily_quotes.trade_volume > ?"</span>, number * <span class="number">1000</span>)</span><br><span class="line"> .order(<span class="string">"daily_quotes.trade_volume DESC"</span>)</span><br><span class="line"> .select(<span class="symbol">:name</span>, <span class="symbol">:code</span>, <span class="string">"daily_quotes.trade_volume"</span>)</span><br><span class="line"> .map { <span class="params">|stock|</span> { <span class="string">"股票名稱"</span> => stock.name, <span class="string">"股票代號"</span> => stock.code, <span class="string">"成交量(張)"</span> => stock.trade_volume / <span class="number">1000</span> } }</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 找出指定日期，漲停的股票</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">self</span>.<span class="title">find_limit_up_stocks</span><span class="params">(date = <span class="literal">nil</span>)</span></span></span><br><span class="line"> latest_transaction_date = find_latest_transaction_dates(<span class="number">2</span>, date)</span><br><span class="line"> candidate_stocks = find_ups_and_downs(latest_transaction_date)</span><br><span class="line"> candidate_stocks.select <span class="keyword">do</span> <span class="params">|key, values|</span></span><br><span class="line"> latest_info = <span class="literal">nil</span></span><br><span class="line"> yesterday_info = <span class="literal">nil</span></span><br><span class="line"> values.each <span class="keyword">do</span> <span class="params">|value|</span></span><br><span class="line"> <span class="keyword">if</span> value.transaction_date == latest_transaction_date[<span class="number">0</span>]</span><br><span class="line"> latest_info = value</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> yesterday_info = value</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">next</span> <span class="keyword">if</span> latest_info.ups_and_downs != <span class="string">"+"</span> <span class="params">||</span> latest_info.closing_price != latest_info.highest_price</span><br><span class="line"></span><br><span class="line"> latest_info.closing_price >= yesterday_info.closing_price * <span class="number">1.095</span></span><br><span class="line"> <span class="keyword">end</span>.keys</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># 找出指定日期，跌停的股票</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">self</span>.<span class="title">find_limit_down_stocks</span><span class="params">(date = <span class="literal">nil</span>)</span></span></span><br><span class="line"> latest_transaction_date = find_latest_transaction_dates(<span class="number">2</span>, date)</span><br><span class="line"> candidate_stocks = find_ups_and_downs(latest_transaction_date)</span><br><span class="line"> candidate_stocks.select <span class="keyword">do</span> <span class="params">|key, values|</span></span><br><span class="line"> latest_info = <span class="literal">nil</span></span><br><span class="line"> yesterday_info = <span class="literal">nil</span></span><br><span class="line"> values.each <span class="keyword">do</span> <span class="params">|value|</span></span><br><span class="line"> <span class="keyword">if</span> value.transaction_date == latest_transaction_date[<span class="number">0</span>]</span><br><span class="line"> latest_info = value</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> yesterday_info = value</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">next</span> <span class="keyword">if</span> latest_info.ups_and_downs != <span class="string">"-"</span> <span class="params">||</span> latest_info.closing_price != latest_info.lowest_price</span><br><span class="line"></span><br><span class="line"> latest_info.closing_price <= yesterday_info.closing_price / <span class="number">1.095</span></span><br><span class="line"> <span class="keyword">end</span>.keys</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> private</span><br><span class="line"></span><br><span class="line"> <span class="comment"># 找出最新 X 天的交易日期</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">self</span>.<span class="title">find_latest_transaction_dates</span><span class="params">(number, date = <span class="literal">nil</span>)</span></span></span><br><span class="line"> <span class="keyword">if</span> date</span><br><span class="line"> DailyQuote.where(<span class="string">"transaction_date <= ?"</span>, date.to_date)</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> DailyQuote</span><br><span class="line"> <span class="keyword">end</span>.select(<span class="symbol">:transaction_date</span>).order(<span class="string">"transaction_date DESC"</span>).distinct.limit(number).pluck(<span class="symbol">:transaction_date</span>)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">self</span>.<span class="title">find_ups_and_downs</span><span class="params">(latest_transaction_date)</span></span></span><br><span class="line"> <span class="keyword">self</span>.joins(<span class="symbol">:daily_quotes</span>)</span><br><span class="line"> .where(<span class="symbol">daily_quotes:</span> { <span class="symbol">transaction_date:</span> latest_transaction_date[-<span class="number">1</span>]..latest_transaction_date[<span class="number">0</span>] })</span><br><span class="line"> .order(<span class="string">"daily_quotes.trade_volume DESC"</span>)</span><br><span class="line"> .select(<span class="symbol">:code</span>, <span class="symbol">:name</span>,</span><br><span class="line"> <span class="string">"daily_quotes.transaction_date"</span>, <span class="string">"daily_quotes.opening_price"</span>, <span class="string">"daily_quotes.closing_price"</span>,</span><br><span class="line"> <span class="string">"daily_quotes.highest_price"</span>, <span class="string">"daily_quotes.lowest_price"</span>, <span class="string">"daily_quotes.ups_and_downs"</span>,</span><br><span class="line"> )</span><br><span class="line"> .group_by { <span class="params">|stock|</span> { <span class="string">"股票名稱"</span> => stock.name, <span class="string">"股票代號"</span> => stock.code } }</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><h2><span id="實際執行畫面">實際執行畫面</span></h2><p><img src="8FSCAhq.png" alt></p><h2><span id="小結">小結</span></h2><p>每當寫完一個技術指標後，如何驗證自己寫的是否正確呢?</p><p>我的做法是去用其他平台，然後看結果是否與我一樣 XD</p><p>有更好的方法或發現有寫錯的地方，歡迎和我說喔</p><p>與「臺灣證券交易所」有關的系列文，這邊告一段落了，後面還有 4 篇鐵人賽文章要寫 (掩面哭，怎還沒結束 QAQ)</p><hr><p>鐵人賽文章連結：<a href="https://ithelp.ithome.com.tw/articles/10273425" target="_blank" rel="noopener">https://ithelp.ithome.com.tw/articles/10273425</a><br>medium 文章連結：<a href="https://link.medium.com/G7zXw5NuTjb" target="_blank" rel="noopener">https://link.medium.com/G7zXw5NuTjb</a><br>本文同步發布於 <a href="http://riverye.com/2021/10/09/Day26-%E7%94%A8-Ruby-on-Rails-%E5%AF%AB%E5%88%86%E6%9E%90%E8%82%A1%E7%A5%A8%E7%9A%84%E6%8A%80%E8%A1%93%E6%8C%87%E6%A8%99/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>在做選股之前，可透過技術指標來分析，像是可以用 KD、均線、量價...等</p>
<h2><span id="說明">說明</span></h2>
<p>這邊要申明下，這邊是以技術交流為主，不會有任何投資建議
</summary>
<category term="鐵人賽" scheme="https://riverye.com/categories/%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
<category term="13th鐵人賽" scheme="https://riverye.com/tags/13th%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
</entry>
<entry>
<title>Day25 - 透過 Rake 自動下載處理臺灣證券交易所的資料</title>
<link href="https://riverye.com/2021/10/08/Day25-%E9%80%8F%E9%81%8E-Rake-%E8%87%AA%E5%8B%95%E4%B8%8B%E8%BC%89%E8%99%95%E7%90%86%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E7%9A%84%E8%B3%87%E6%96%99/"/>
<id>https://riverye.com/2021/10/08/Day25-透過-Rake-自動下載處理臺灣證券交易所的資料/</id>
<published>2021-10-07T16:00:00.000Z</published>
<updated>2022-12-21T14:34:01.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>已經能從「臺灣證券交易所」抓資料、存入 DB，接下來要做自動化處理</p><h2><span id="說明">說明</span></h2><p>由於我電腦沒有 24 小時開著，加上專案也不會一直開著，需要時才會手動下 Rake 執行，因此沒有寫在 schedule，這部分可因需求，自行調整，這邊以 Rake 為例</p><h2><span id="實作">實作</span></h2><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># lib/tasks/twse.rake</span></span><br><span class="line"></span><br><span class="line">namespace <span class="symbol">:twse</span> <span class="keyword">do</span></span><br><span class="line"></span><br><span class="line"> desc <span class="string">"auto execute all"</span></span><br><span class="line"> task <span class="symbol">auto:</span> <span class="symbol">:environment</span> <span class="keyword">do</span></span><br><span class="line"> Rake::Task[<span class="string">"twse:download_all"</span>].invoke</span><br><span class="line"> Rake::Task[<span class="string">"twse:save_all"</span>].invoke</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> desc <span class="string">"download twse ALLBUT0999、TWT49U data"</span></span><br><span class="line"> task <span class="symbol">download_all:</span> <span class="symbol">:environment</span> <span class="keyword">do</span></span><br><span class="line"> Rake::Task[<span class="string">"twse:download_file"</span>].invoke</span><br><span class="line"> Rake::Task[<span class="string">"twse:download_dr_file"</span>].invoke</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> desc <span class="string">"download twse ALLBUT0999 每日收盤行情(全部(不含權證、牛熊證、可展延牛熊證)) data"</span></span><br><span class="line"> task <span class="symbol">download_file:</span> <span class="symbol">:environment</span> <span class="keyword">do</span></span><br><span class="line"> Twse::Allbut0999::Download.new.execute</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> desc <span class="string">"download twse TWT49U 除權除息計算結果表 data"</span></span><br><span class="line"> task <span class="symbol">download_dr_file:</span> <span class="symbol">:environment</span> <span class="keyword">do</span></span><br><span class="line"> Twse::Twt49u::Download.new.execute</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> desc <span class="string">"ALLBUT0999、TWT49U CVS save to database"</span></span><br><span class="line"> task <span class="symbol">save_all:</span> <span class="symbol">:environment</span> <span class="keyword">do</span></span><br><span class="line"> Rake::Task[<span class="string">"twse:save_dr_to_db"</span>].invoke</span><br><span class="line"> Rake::Task[<span class="string">"twse:save_to_db"</span>].invoke</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> desc <span class="string">"ALLBUT0999 CVS save to database"</span></span><br><span class="line"> task <span class="symbol">save_to_db:</span> <span class="symbol">:environment</span> <span class="keyword">do</span></span><br><span class="line"> Twse::Allbut0999::SaveToDb.new.execute</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> desc <span class="string">"TWT49U CVS save to database"</span></span><br><span class="line"> task <span class="symbol">save_dr_to_db:</span> <span class="symbol">:environment</span> <span class="keyword">do</span></span><br><span class="line"> Twse::Twt49u::SaveToDb.new.execute</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><p>在 Terminal 輸入</p><p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">rake twse:auto</span><br><span class="line"></span><br><span class="line"><span class="comment"># or</span></span><br><span class="line"></span><br><span class="line">bundle <span class="built_in">exec</span> rake twse:auto</span><br></pre></td></tr></table></figure></p><h2><span id="實際執行畫面">實際執行畫面</span></h2><p><img src="g9jp0O0.png" alt></p><h2><span id="小結">小結</span></h2><p>現在只需要手動下一行 code 就會自動下載、存入 DB、上傳到 GitHub，非常的方便</p><p>工程師的特性，發現要做重複的事情時，會想辦法做成自動化處理~</p><hr><p>鐵人賽文章連結：<a href="https://ithelp.ithome.com.tw/articles/10273391" target="_blank" rel="noopener">https://ithelp.ithome.com.tw/articles/10273391</a><br>medium 文章連結：<a href="https://link.medium.com/wzKY4dNuTjb" target="_blank" rel="noopener">https://link.medium.com/wzKY4dNuTjb</a><br>本文同步發布於 <a href="http://riverye.com/2021/10/08/Day25-%E9%80%8F%E9%81%8E-Rake-%E8%87%AA%E5%8B%95%E4%B8%8B%E8%BC%89%E8%99%95%E7%90%86%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E7%9A%84%E8%B3%87%E6%96%99/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>已經能從「臺灣證券交易所」抓資料、存入 DB，接下來要做自動化處理</p>
<h2><span id="說明">說明</span></h2>
<p>由於我電腦沒有 24 小時開著，加上專案也不會一直開著，需要
</summary>
<category term="鐵人賽" scheme="https://riverye.com/categories/%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
<category term="13th鐵人賽" scheme="https://riverye.com/tags/13th%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
</entry>
<entry>
<title>Day24 - 將臺灣證券交易所的除權除息計算結果表存入 DB</title>
<link href="https://riverye.com/2021/10/07/Day24-%E5%B0%87%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E7%9A%84%E9%99%A4%E6%AC%8A%E9%99%A4%E6%81%AF%E8%A8%88%E7%AE%97%E7%B5%90%E6%9E%9C%E8%A1%A8%E5%AD%98%E5%85%A5-DB/"/>
<id>https://riverye.com/2021/10/07/Day24-將臺灣證券交易所的除權除息計算結果表存入-DB/</id>
<published>2021-10-06T16:00:00.000Z</published>
<updated>2022-12-21T14:32:09.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>前面已經知道如何抓「臺灣證券交易所」的除權除息計算結果表 CSV 檔，接下來要處理資料，並存入 DB</p><h2><span id="說明">說明</span></h2><p>需要考量的情境，與<a href="https://ithelp.ithome.com.tw/articles/10272899" target="_blank" rel="noopener">前一篇</a>是一樣的 (描述越來越精簡 XD)</p><h2><span id="實作">實作</span></h2><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># app/features/twse/twt_49u/save_to_db.rb</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">module</span> <span class="title">Twse::Twt49u</span></span></span><br><span class="line"> <span class="class"><span class="keyword">class</span> <span class="title">SaveToDb</span></span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">include</span> Twse::Helpers</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">execute</span></span></span><br><span class="line"> start_time = Time.current</span><br><span class="line"> puts <span class="string">"<span class="subst">#{<span class="keyword">self</span>.<span class="keyword">class</span>}</span>, start_time: <span class="subst">#{start_time.to_s}</span>"</span></span><br><span class="line"></span><br><span class="line"> latest_data_date = find_latest_data_date</span><br><span class="line"> <span class="keyword">return</span> puts <span class="string">"<span class="subst">#{<span class="keyword">self</span>.<span class="keyword">class</span>}</span>, 已經是最新的資料"</span> <span class="keyword">if</span> latest_data_date == start_time.to_date</span><br><span class="line"></span><br><span class="line"> is_linux = <span class="string">`uname -a`</span>[<span class="regexp">/Linux/</span>].present?</span><br><span class="line"> file_paths = Dir[<span class="string">"data/twse/TWT49U/*"</span>]</span><br><span class="line"> file_paths.each <span class="keyword">do</span> <span class="params">|file_path|</span></span><br><span class="line"> rows = decode_data(file_path, is_linux)</span><br><span class="line"> first_year, end_year = time_range(rows)</span><br><span class="line"> row_index, end_index = rows_range(rows)</span><br><span class="line"> all_rows, data_date_infos = filter_rows(rows, row_index, end_index)</span><br><span class="line"> <span class="keyword">next</span> <span class="keyword">if</span> not_process?(data_date_infos, latest_data_date)</span><br><span class="line"></span><br><span class="line"> filtered_stocks = filter_by_stocks(all_rows)</span><br><span class="line"> Stock.import(filtered_stocks) <span class="keyword">if</span> filtered_stocks.present?</span><br><span class="line"></span><br><span class="line"> import_ex_stocks(all_rows)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> puts <span class="string">"<span class="subst">#{<span class="keyword">self</span>.<span class="keyword">class</span>}</span>, done_time:<span class="subst">#{Time.current}</span>, <span class="subst">#{(Time.current - start_time).to_s}</span> sec"</span></span><br><span class="line"> <span class="keyword">rescue</span> StandardError => e</span><br><span class="line"> puts <span class="string">"errors: <span class="subst">#{e.inspect}</span>, <span class="subst">#{e.backtrace}</span>"</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> private</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">find_latest_data_date</span></span></span><br><span class="line"> ExStock.latest_data_date</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">time_range</span><span class="params">(rows)</span></span></span><br><span class="line"> first_year, first_month, first_day, end_year, end_month, end_day = rows[<span class="number">0</span>].scan(<span class="regexp">/\d+/</span>)</span><br><span class="line"> [first_year, end_year].each_with_index <span class="keyword">do</span> <span class="params">|year, index|</span></span><br><span class="line"> year = <span class="string">"20"</span> + (year.to_i + <span class="number">11</span>).to_s[<span class="number">1</span>..<span class="number">2</span>]</span><br><span class="line"> <span class="keyword">if</span> index.zero?</span><br><span class="line"> first_year = year</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> end_year =year</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> [first_year, end_year]</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">rows_range</span><span class="params">(rows)</span></span></span><br><span class="line"> row_index = <span class="literal">nil</span></span><br><span class="line"> rows.each_with_index <span class="keyword">do</span> <span class="params">|row, index|</span></span><br><span class="line"> <span class="keyword">if</span> row.<span class="keyword">include</span>?(<span class="string">"資料日期"</span>)</span><br><span class="line"> row_index = index</span><br><span class="line"> <span class="keyword">break</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> end_index = <span class="literal">nil</span></span><br><span class="line"> rows.each_with_index <span class="keyword">do</span> <span class="params">|row, index|</span></span><br><span class="line"> <span class="keyword">if</span> row.<span class="keyword">include</span>?(<span class="string">"公式"</span>)</span><br><span class="line"> end_index = index</span><br><span class="line"> <span class="keyword">break</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> [row_index, end_index]</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">filter_rows</span><span class="params">(rows, row_index, end_index)</span></span></span><br><span class="line"> all_rows = []</span><br><span class="line"> data_date_infos = []</span><br><span class="line"> rows[(row_index + <span class="number">1</span>)..(end_index - <span class="number">1</span>)].each <span class="keyword">do</span> <span class="params">|row_string|</span></span><br><span class="line"> row_item = []</span><br><span class="line"> row_string.split(<span class="string">'",'</span>).each { <span class="params">|row|</span> row_item << row.gsub(<span class="regexp">/[=|,|"]/</span>, <span class="string">''</span>) }</span><br><span class="line"></span><br><span class="line"> year, month, day = row_item[<span class="number">0</span>].scan(<span class="regexp">/\d+/</span>)</span><br><span class="line"> year = <span class="string">"20"</span> + (year.to_i + <span class="number">11</span>).to_s[<span class="number">1</span>..<span class="number">2</span>]</span><br><span class="line"> data_date = year + month + day</span><br><span class="line"> row_item[<span class="number">0</span>] = data_date.to_date</span><br><span class="line"> all_rows << row_item</span><br><span class="line"> data_date_infos << row_item[<span class="number">0</span>]</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">return</span> [all_rows, data_date_infos]</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">not_process?</span><span class="params">(data_date_infos, latest_data_date)</span></span></span><br><span class="line"> data_date_infos.uniq!</span><br><span class="line"> latest_data_date.present? && data_date_infos.max <= latest_data_date</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">filter_by_stocks</span><span class="params">(all_rows)</span></span></span><br><span class="line"> stock_infos = []</span><br><span class="line"> all_rows.each { <span class="params">|rows|</span> stock_infos << { <span class="symbol">code:</span> rows[<span class="number">1</span>], <span class="symbol">name:</span> rows[<span class="number">2</span>] } }</span><br><span class="line"> stocks = Stock.all.select(<span class="symbol">:code</span>).index_by(&<span class="symbol">:code</span>)</span><br><span class="line"></span><br><span class="line"> need_create_stocks = []</span><br><span class="line"> stock_infos.each <span class="keyword">do</span> <span class="params">|stock_info|</span></span><br><span class="line"> stock = stocks[stock_info[<span class="symbol">:code</span>]]</span><br><span class="line"> <span class="keyword">next</span> <span class="keyword">if</span> stock</span><br><span class="line"> <span class="keyword">next</span> <span class="keyword">if</span> need_create_stocks.any? { <span class="params">|item|</span> item.code == stock_info[<span class="symbol">:code</span>] }</span><br><span class="line"></span><br><span class="line"> need_create_stocks << Stock.new(<span class="symbol">code:</span> stock_info[<span class="symbol">:code</span>], <span class="symbol">name:</span> stock_info[<span class="symbol">:name</span>])</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> need_create_stocks</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">import_ex_stocks</span><span class="params">(all_rows)</span></span></span><br><span class="line"> stocks = Stock.all.select(<span class="symbol">:code</span>).index_by(&<span class="symbol">:code</span>)</span><br><span class="line"></span><br><span class="line"> need_create_ex_stocks = []</span><br><span class="line"> all_rows.each <span class="keyword">do</span> <span class="params">|row|</span></span><br><span class="line"> stock = stocks[row[<span class="number">1</span>]]</span><br><span class="line"></span><br><span class="line"> need_create_ex_stocks << stock.exs.new(</span><br><span class="line"> <span class="symbol">data_date:</span> row[<span class="number">0</span>],</span><br><span class="line"> <span class="symbol">closing_price_before:</span> row[<span class="number">3</span>],</span><br><span class="line"> <span class="symbol">reference_price:</span> row[<span class="number">4</span>],</span><br><span class="line"> <span class="symbol">dr_value:</span> row[<span class="number">5</span>],</span><br><span class="line"> <span class="symbol">dividend_right:</span> ExStock::DIVIDEND_RIGHT[row[<span class="number">6</span>]],</span><br><span class="line"> <span class="symbol">limit_up:</span> row[<span class="number">7</span>],</span><br><span class="line"> <span class="symbol">limit_down:</span> row[<span class="number">8</span>],</span><br><span class="line"> <span class="symbol">opening_reference_price:</span> row[<span class="number">9</span>],</span><br><span class="line"> <span class="symbol">ex_dividend_reference_price:</span> row[<span class="number">10</span>],</span><br><span class="line"> <span class="symbol">reporting_day:</span> row[<span class="number">12</span>],</span><br><span class="line"> <span class="symbol">price_book:</span> row[<span class="number">13</span>],</span><br><span class="line"> <span class="symbol">eps:</span> row[<span class="number">14</span>],</span><br><span class="line"> )</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> ExStock.import(need_create_ex_stocks) <span class="keyword">if</span> need_create_ex_stocks.present?</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><h2><span id="檢查是否有存入-db">檢查是否有存入 DB</span></h2><p><img src="fTpuAHD.png" alt></p><h2><span id="小結">小結</span></h2><p>我猜看到這，已經不少人看不懂或沒在看了 (笑)</p><p>說明越寫越精簡，想說 code 已經直接貼出來了，直接看 code 比較快，若發現有更好的寫法，歡迎留言和我說~</p><hr><p>鐵人賽文章連結：<a href="https://ithelp.ithome.com.tw/articles/10272913" target="_blank" rel="noopener">https://ithelp.ithome.com.tw/articles/10272913</a><br>medium 文章連結：<a href="https://link.medium.com/VfJfmvMuTjb" target="_blank" rel="noopener">https://link.medium.com/VfJfmvMuTjb</a><br>本文同步發布於 <a href="http://riverye.com/2021/10/07/Day24-%E5%B0%87%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E7%9A%84%E9%99%A4%E6%AC%8A%E9%99%A4%E6%81%AF%E8%A8%88%E7%AE%97%E7%B5%90%E6%9E%9C%E8%A1%A8%E5%AD%98%E5%85%A5-DB/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>前面已經知道如何抓「臺灣證券交易所」的除權除息計算結果表 CSV 檔，接下來要處理資料，並存入 DB</p>
<h2><span id="說明">說明</span></h2>
<p>需要考量的情境，與<a h
</summary>
<category term="鐵人賽" scheme="https://riverye.com/categories/%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
<category term="13th鐵人賽" scheme="https://riverye.com/tags/13th%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
</entry>
<entry>
<title>Day23 - 將臺灣證券交易所的每日收盤行情存入 DB</title>
<link href="https://riverye.com/2021/10/06/Day23-%E5%B0%87%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E7%9A%84%E6%AF%8F%E6%97%A5%E6%94%B6%E7%9B%A4%E8%A1%8C%E6%83%85%E5%AD%98%E5%85%A5-DB/"/>
<id>https://riverye.com/2021/10/06/Day23-將臺灣證券交易所的每日收盤行情存入-DB/</id>
<published>2021-10-05T16:00:00.000Z</published>
<updated>2022-12-21T14:33:52.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>前面已經知道如何抓「臺灣證券交易所」的每日收盤行情 CSV 檔，接下來要處理資料，並存入 DB</p><h2><span id="說明">說明</span></h2><p>在處理過程中，需要考量，可能會有新上市的公司，若有的話，要建立 <code>Stock</code>，至於已經下市的公司，則應該要軟刪除 (這部分還沒做，可自行研究)</p><p>同一天的資料，若執行多次，應該要判斷是否已在 DB 建立過，不應該重複建立</p><h2><span id="實作">實作</span></h2><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># app/features/twse/allbut_0999/save_to_db.rb</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">module</span> <span class="title">Twse::Allbut0999</span></span></span><br><span class="line"> <span class="class"><span class="keyword">class</span> <span class="title">SaveToDb</span></span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">include</span> Twse::Helpers</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">execute</span></span></span><br><span class="line"> start_time = Time.current</span><br><span class="line"> puts <span class="string">"<span class="subst">#{<span class="keyword">self</span>.<span class="keyword">class</span>}</span>, start_time: <span class="subst">#{start_time.to_s}</span>"</span></span><br><span class="line"></span><br><span class="line"> end_transaction_date = find_latest_transaction_date</span><br><span class="line"> <span class="keyword">return</span> puts <span class="string">"<span class="subst">#{<span class="keyword">self</span>.<span class="keyword">class</span>}</span>, 已經是最新的資料"</span> <span class="keyword">if</span> end_transaction_date == start_time</span><br><span class="line"></span><br><span class="line"> is_linux = <span class="string">`uname -a`</span>[<span class="regexp">/Linux/</span>].present?</span><br><span class="line"> file_paths = Dir[<span class="string">"data/twse/ALLBUT0999/*/*/*"</span>]</span><br><span class="line"> file_paths.each <span class="keyword">do</span> <span class="params">|file_path|</span></span><br><span class="line"> rows = decode_data(file_path, is_linux)</span><br><span class="line"> <span class="keyword">next</span> <span class="keyword">if</span> not_process?(rows, end_transaction_date)</span><br><span class="line"></span><br><span class="line"> row_index = find_rows_index(rows)</span><br><span class="line"> all_rows = filter_rows(rows, row_index)</span><br><span class="line"> filtered_stocks = filter_by_stocks(all_rows)</span><br><span class="line"> Stock.import(filtered_stocks) <span class="keyword">if</span> filtered_stocks.present?</span><br><span class="line"></span><br><span class="line"> import_daily_quotes(all_rows)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> puts <span class="string">"<span class="subst">#{<span class="keyword">self</span>.<span class="keyword">class</span>}</span>, done_time:<span class="subst">#{Time.current}</span>, <span class="subst">#{(Time.current - start_time).to_s}</span> sec"</span></span><br><span class="line"> <span class="keyword">rescue</span> StandardError => e</span><br><span class="line"> puts <span class="string">"errors: <span class="subst">#{e.inspect}</span>, <span class="subst">#{e.backtrace}</span>"</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> private</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">find_latest_transaction_date</span></span></span><br><span class="line"> DailyQuote.latest_transaction_date</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">not_process?</span><span class="params">(rows, end_transaction_date)</span></span></span><br><span class="line"> year, month, day = rows[<span class="number">0</span>].scan(<span class="regexp">/\d+/</span>)</span><br><span class="line"> year = <span class="string">"20"</span> + (year.to_i + <span class="number">11</span>).to_s[<span class="number">1</span>..<span class="number">2</span>]</span><br><span class="line"> @file_date = year + month + day</span><br><span class="line"> end_transaction_date.present? && @file_date.to_date <= end_transaction_date</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">find_rows_index</span><span class="params">(rows)</span></span></span><br><span class="line"> row_index = <span class="literal">nil</span></span><br><span class="line"> rows.each_with_index { <span class="params">|row, index|</span> row_index = index <span class="keyword">if</span> row.<span class="keyword">include</span>?(<span class="string">"證券代號"</span>) }</span><br><span class="line"> row_index</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">filter_rows</span><span class="params">(rows, row_index)</span></span></span><br><span class="line"> all_rows = []</span><br><span class="line"> rows[(row_index + <span class="number">1</span>)..-<span class="number">1</span>].each <span class="keyword">do</span> <span class="params">|row_string|</span></span><br><span class="line"> row_item = []</span><br><span class="line"> row_string.split(<span class="string">',"'</span>).each { <span class="params">|row|</span> row_item << row.gsub(<span class="regexp">/[=|,|"]/</span>, <span class="string">''</span>) }</span><br><span class="line"> all_rows << row_item</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> all_rows</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">filter_by_stocks</span><span class="params">(all_rows)</span></span></span><br><span class="line"> stock_infos = []</span><br><span class="line"> all_rows.each { <span class="params">|rows|</span> stock_infos << { <span class="symbol">code:</span> rows[<span class="number">0</span>], <span class="symbol">name:</span> rows[<span class="number">1</span>] } }</span><br><span class="line"> stocks = Stock.all.select(<span class="symbol">:code</span>).index_by(&<span class="symbol">:code</span>)</span><br><span class="line"></span><br><span class="line"> need_create_stocks = []</span><br><span class="line"> stock_infos.each <span class="keyword">do</span> <span class="params">|stock_info|</span></span><br><span class="line"> stock = stocks[stock_info[<span class="symbol">:code</span>]]</span><br><span class="line"> <span class="keyword">next</span> <span class="keyword">if</span> stock</span><br><span class="line"></span><br><span class="line"> need_create_stocks << Stock.new(<span class="symbol">code:</span> stock_info[<span class="symbol">:code</span>], <span class="symbol">name:</span> stock_info[<span class="symbol">:name</span>])</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> need_create_stocks</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">import_daily_quotes</span><span class="params">(all_rows)</span></span></span><br><span class="line"> stocks = Stock.all.select(<span class="symbol">:code</span>).index_by(&<span class="symbol">:code</span>)</span><br><span class="line"></span><br><span class="line"> need_create_daily_quotes = []</span><br><span class="line"> all_rows.each <span class="keyword">do</span> <span class="params">|row|</span></span><br><span class="line"> stock = stocks[row[<span class="number">0</span>]]</span><br><span class="line"></span><br><span class="line"> need_create_daily_quotes << stock.daily_quotes.new(</span><br><span class="line"> <span class="symbol">transaction_date:</span> @file_date.to_date,</span><br><span class="line"> <span class="symbol">trade_volume:</span> row[<span class="number">2</span>],</span><br><span class="line"> <span class="symbol">number_of_transactions:</span> row[<span class="number">3</span>],</span><br><span class="line"> <span class="symbol">trade_price:</span> row[<span class="number">4</span>],</span><br><span class="line"> <span class="symbol">opening_price:</span> row[<span class="number">5</span>],</span><br><span class="line"> <span class="symbol">highest_price:</span> row[<span class="number">6</span>],</span><br><span class="line"> <span class="symbol">lowest_price:</span> row[<span class="number">7</span>],</span><br><span class="line"> <span class="symbol">closing_price:</span> row[<span class="number">8</span>],</span><br><span class="line"> <span class="symbol">ups_and_downs:</span> row[<span class="number">9</span>],</span><br><span class="line"> <span class="symbol">price_difference:</span> row[<span class="number">10</span>],</span><br><span class="line"> <span class="symbol">last_best_bid_price:</span> row[<span class="number">11</span>],</span><br><span class="line"> <span class="symbol">last_best_bid_volume:</span> row[<span class="number">12</span>],</span><br><span class="line"> <span class="symbol">last_best_ask_price:</span> row[<span class="number">13</span>],</span><br><span class="line"> <span class="symbol">last_best_ask_volume:</span> row[<span class="number">14</span>],</span><br><span class="line"> <span class="symbol">price_earning_ratio:</span> row[<span class="number">15</span>],</span><br><span class="line"> )</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> DailyQuote.import(need_create_daily_quotes) <span class="keyword">if</span> need_create_daily_quotes.present?</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><h2><span id="檢查是否有存入-db">檢查是否有存入 DB</span></h2><p><img src="0BmUTWQ.png" alt></p><h2><span id="小結">小結</span></h2><p>現在看別人寫好的 code ，應該會覺得蠻簡單的，當下自己在做的時候，並沒有範例的 code 可以參考，邊摸索邊做出來的，過程的酸甜苦辣只有自己知道，唯有自己做過，才會知道~</p><hr><p>鐵人賽文章連結：<a href="https://ithelp.ithome.com.tw/articles/10272899" target="_blank" rel="noopener">https://ithelp.ithome.com.tw/articles/10272899</a><br>medium 文章連結：<a href="https://link.medium.com/HmmocOLuTjb" target="_blank" rel="noopener">https://link.medium.com/HmmocOLuTjb</a><br>本文同步發布於 <a href="http://riverye.com/2021/10/06/Day23-%E5%B0%87%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E7%9A%84%E6%AF%8F%E6%97%A5%E6%94%B6%E7%9B%A4%E8%A1%8C%E6%83%85%E5%AD%98%E5%85%A5-DB/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>前面已經知道如何抓「臺灣證券交易所」的每日收盤行情 CSV 檔，接下來要處理資料，並存入 DB</p>
<h2><span id="說明">說明</span></h2>
<p>在處理過程中，需要考量，可能會有
</summary>
<category term="鐵人賽" scheme="https://riverye.com/categories/%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
<category term="13th鐵人賽" scheme="https://riverye.com/tags/13th%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
</entry>
<entry>
<title>Day22 - 用 Ruby on Rails 處理臺灣證券交易所資料-DB 設計</title>
<link href="https://riverye.com/2021/10/05/Day22-%E7%94%A8-Ruby-on-Rails-%E8%99%95%E7%90%86%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E8%B3%87%E6%96%99-DB-%E8%A8%AD%E8%A8%88/"/>
<id>https://riverye.com/2021/10/05/Day22-用-Ruby-on-Rails-處理臺灣證券交易所資料-DB-設計/</id>
<published>2021-10-04T16:00:00.000Z</published>
<updated>2022-09-23T08:45:10.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>有了前 2 篇從「臺灣證券交易所」取得 CSV 檔後，接著要把資料存入 DB，在存入前，需要先有 DB，本篇以 DB 設計為主</p><h2><span id="說明">說明</span></h2><p>預期會有</p><ol><li>一個 Model 紀錄股票名稱、代號 (<code>Stock</code>)</li><li>一個 Model 紀錄每日收盤行情相關資訊 (<code>DailyQuote</code>)</li><li>一個 Model 紀錄除權除息計算結果表 (<code>ExStock</code>)</li><li><code>Stock</code> 與 <code>DailyQuote</code> 為一對多關聯</li><li><code>Stock</code> 與 <code>ExStock</code> 為一對多關聯</li></ol><h2><span id="實作">實作</span></h2><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">CreateStocks</span> < ActiveRecord::Migration[6.1]</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">change</span></span></span><br><span class="line"> create_table <span class="symbol">:stocks</span> <span class="keyword">do</span> <span class="params">|t|</span></span><br><span class="line"> t.string <span class="symbol">:name</span>, <span class="symbol">null:</span> <span class="literal">false</span></span><br><span class="line"> t.string <span class="symbol">:code</span>, <span class="symbol">null:</span> <span class="literal">false</span></span><br><span class="line"> t.datetime <span class="symbol">:deleted_at</span></span><br><span class="line"></span><br><span class="line"> t.timestamps</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> add_index <span class="symbol">:stocks</span>, <span class="symbol">:code</span>, <span class="symbol">unique:</span> <span class="literal">true</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">CreateDailyQuotes</span> < ActiveRecord::Migration[6.1]</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">change</span></span></span><br><span class="line"> create_table <span class="symbol">:daily_quotes</span> <span class="keyword">do</span> <span class="params">|t|</span></span><br><span class="line"> t.string <span class="symbol">:code</span>, <span class="symbol">null:</span> <span class="literal">false</span></span><br><span class="line"> t.date <span class="symbol">:transaction_date</span>, <span class="symbol">null:</span> <span class="literal">false</span> <span class="comment"># 收盤日期</span></span><br><span class="line"> t.bigint <span class="symbol">:trade_volume</span> <span class="comment"># 成交股數</span></span><br><span class="line"> t.bigint <span class="symbol">:number_of_transactions</span> <span class="comment"># 成交筆數</span></span><br><span class="line"> t.bigint <span class="symbol">:trade_price</span> <span class="comment"># 成交金額</span></span><br><span class="line"> t.float <span class="symbol">:opening_price</span> <span class="comment"># 開盤價</span></span><br><span class="line"> t.float <span class="symbol">:highest_price</span> <span class="comment"># 最高價</span></span><br><span class="line"> t.float <span class="symbol">:lowest_price</span> <span class="comment"># 最低價</span></span><br><span class="line"> t.float <span class="symbol">:closing_price</span> <span class="comment"># 收盤價</span></span><br><span class="line"> t.string <span class="symbol">:ups_and_downs</span> <span class="comment"># 漲跌</span></span><br><span class="line"> t.float <span class="symbol">:price_difference</span> <span class="comment"># 價差</span></span><br><span class="line"> t.float <span class="symbol">:last_best_bid_price</span> <span class="comment"># 最後揭示買價</span></span><br><span class="line"> t.bigint <span class="symbol">:last_best_bid_volume</span> <span class="comment"># 最後揭示買量</span></span><br><span class="line"> t.float <span class="symbol">:last_best_ask_price</span> <span class="comment"># 最後揭示賣價</span></span><br><span class="line"> t.bigint <span class="symbol">:last_best_ask_volume</span> <span class="comment"># 最後揭示賣量</span></span><br><span class="line"> t.float <span class="symbol">:price_earning_ratio</span> <span class="comment"># 本益比</span></span><br><span class="line"></span><br><span class="line"> t.timestamps</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> add_index <span class="symbol">:daily_quotes</span>, %i[code transaction_date], <span class="symbol">unique:</span> <span class="literal">true</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">CreateExStocks</span> < ActiveRecord::Migration[6.1]</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">change</span></span></span><br><span class="line"> create_table <span class="symbol">:ex_stocks</span> <span class="keyword">do</span> <span class="params">|t|</span></span><br><span class="line"> t.string <span class="symbol">:code</span>, <span class="symbol">null:</span> <span class="literal">false</span></span><br><span class="line"> t.date <span class="symbol">:data_date</span>, <span class="symbol">null:</span> <span class="literal">false</span> <span class="comment"># 資料日期</span></span><br><span class="line"> t.float <span class="symbol">:closing_price_before</span>, <span class="symbol">null:</span> <span class="literal">false</span> <span class="comment"># 除權息前收盤價</span></span><br><span class="line"> t.float <span class="symbol">:reference_price</span>, <span class="symbol">null:</span> <span class="literal">false</span> <span class="comment"># 除權息參考價</span></span><br><span class="line"> t.float <span class="symbol">:dr_value</span>, <span class="symbol">null:</span> <span class="literal">false</span> <span class="comment"># 權值+息值</span></span><br><span class="line"> t.integer <span class="symbol">:dividend_right</span>, <span class="symbol">null:</span> <span class="literal">false</span> <span class="comment"># 權/息</span></span><br><span class="line"> t.float <span class="symbol">:limit_up</span>, <span class="symbol">null:</span> <span class="literal">false</span> <span class="comment"># 漲停價格</span></span><br><span class="line"> t.float <span class="symbol">:limit_down</span>, <span class="symbol">null:</span> <span class="literal">false</span> <span class="comment"># 跌停價格</span></span><br><span class="line"> t.float <span class="symbol">:opening_reference_price</span>, <span class="symbol">null:</span> <span class="literal">false</span> <span class="comment"># 開盤競價基準</span></span><br><span class="line"> t.float <span class="symbol">:ex_dividend_reference_price</span>, <span class="symbol">null:</span> <span class="literal">false</span> <span class="comment"># 減除股利參考價</span></span><br><span class="line"> t.string <span class="symbol">:reporting_day</span> <span class="comment"># 最近一次申報資料 季別/日期</span></span><br><span class="line"> t.float <span class="symbol">:price_book</span> <span class="comment"># 最近一次申報每股 (單位)淨值</span></span><br><span class="line"> t.float <span class="symbol">:eps</span> <span class="comment"># 最近一次申報每股 (單位)盈餘</span></span><br><span class="line"></span><br><span class="line"> t.timestamps</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> add_index <span class="symbol">:ex_stocks</span>, %i[code data_date], <span class="symbol">unique:</span> <span class="literal">true</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># app/models/stock.rb</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Stock</span> < ApplicationRecord</span></span><br><span class="line"> validates <span class="symbol">:name</span>, <span class="symbol">:code</span>, <span class="symbol">presence:</span> <span class="literal">true</span></span><br><span class="line"> validates <span class="symbol">:code</span>, <span class="symbol">uniqueness:</span> <span class="literal">true</span></span><br><span class="line"></span><br><span class="line"> has_many <span class="symbol">:daily_quotes</span>, <span class="symbol">foreign_key:</span> <span class="symbol">:code</span>, <span class="symbol">primary_key:</span> <span class="symbol">:code</span>, <span class="symbol">dependent:</span> <span class="symbol">:destroy</span></span><br><span class="line"> has_many <span class="symbol">:exs</span>, <span class="symbol">class_name:</span> <span class="string">"ExStock"</span>, <span class="symbol">foreign_key:</span> <span class="symbol">:code</span>, <span class="symbol">primary_key:</span> <span class="symbol">:code</span>, <span class="symbol">dependent:</span> <span class="symbol">:destroy</span></span><br><span class="line"> </span><br><span class="line"> scope <span class="symbol">:latest_transaction_date</span>, -> (date = <span class="literal">nil</span>) <span class="keyword">do</span></span><br><span class="line"> date = date ? date.to_date : DailyQuote.latest_transaction_date</span><br><span class="line"> <span class="keyword">self</span>.joins(<span class="symbol">:daily_quotes</span>).where(<span class="symbol">daily_quotes:</span> { <span class="symbol">transaction_date:</span> date })</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># app/models/daily_quote.rb</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">DailyQuote</span> < ApplicationRecord</span></span><br><span class="line"> acts_as_paranoid</span><br><span class="line"></span><br><span class="line"> validates <span class="symbol">:code</span>, <span class="symbol">:transaction_date</span>, <span class="symbol">presence:</span> <span class="literal">true</span></span><br><span class="line"> validates <span class="symbol">:code</span>, <span class="symbol">uniqueness:</span> { <span class="symbol">scope:</span> <span class="symbol">:transaction_date</span>,</span><br><span class="line"> <span class="symbol">message:</span> <span class="string">"該收盤日期已有紀錄"</span> }</span><br><span class="line"></span><br><span class="line"> belongs_to <span class="symbol">:stock</span>, <span class="symbol">foreign_key:</span> <span class="symbol">:code</span>, <span class="symbol">primary_key:</span> <span class="symbol">:code</span></span><br><span class="line"></span><br><span class="line"> scope <span class="symbol">:latest_transaction_date</span>, -> { maximum(<span class="symbol">:transaction_date</span>) }</span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># app/models/ex_stock.rb</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">ExStock</span> < ApplicationRecord</span></span><br><span class="line"> validates <span class="symbol">:code</span>, <span class="symbol">:data_date</span>, <span class="symbol">:closing_price_before</span>, <span class="symbol">:reference_price</span>, <span class="symbol">:dr_value</span>,</span><br><span class="line"> <span class="symbol">:dividend_right</span>, <span class="symbol">:limit_up</span>, <span class="symbol">:limit_down</span>, <span class="symbol">:opening_reference_price</span>, <span class="symbol">:ex_dividend_reference_price</span>,</span><br><span class="line"> <span class="symbol">presence:</span> <span class="literal">true</span></span><br><span class="line"> validates <span class="symbol">:code</span>, <span class="symbol">uniqueness:</span> { <span class="symbol">scope:</span> <span class="symbol">:data_date</span>, <span class="symbol">message:</span> <span class="string">"該資料日期已有紀錄"</span> }</span><br><span class="line"></span><br><span class="line"> belongs_to <span class="symbol">:stock</span>, <span class="symbol">foreign_key:</span> <span class="symbol">:code</span>, <span class="symbol">primary_key:</span> <span class="symbol">:code</span></span><br><span class="line"></span><br><span class="line"> DIVIDEND_RIGHT = {</span><br><span class="line"> <span class="string">"息"</span> => <span class="string">"xd"</span>,</span><br><span class="line"> <span class="string">"權"</span> => <span class="string">"xr"</span>,</span><br><span class="line"> <span class="string">"權息"</span> => <span class="string">"dr"</span>,</span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> enum <span class="symbol">dividend_right:</span> {</span><br><span class="line"> <span class="symbol">xd:</span> <span class="number">0</span>, <span class="comment"># 除息 (Exclude Dividend)</span></span><br><span class="line"> <span class="symbol">xr:</span> <span class="number">1</span>, <span class="comment"># 除權 (Exclude Right)</span></span><br><span class="line"> <span class="symbol">dr:</span> <span class="number">2</span>, <span class="comment"># 除權除息(同時) (DR (Dividend + Right))</span></span><br><span class="line"> }</span><br><span class="line"></span><br><span class="line"> scope <span class="symbol">:latest_data_date</span>, -> { maximum(<span class="symbol">:data_date</span>) }</span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><h2><span id="小結">小結</span></h2><p>設計建立好 DB 後，接下來的兩篇會示範把前兩篇抓下來的資料存到 DB~</p><hr><p>鐵人賽文章連結：<a href="https://ithelp.ithome.com.tw/articles/10272854" target="_blank" rel="noopener">https://ithelp.ithome.com.tw/articles/10272854</a><br>medium 文章連結：<a href="https://link.medium.com/s8TWR4KuTjb" target="_blank" rel="noopener">https://link.medium.com/s8TWR4KuTjb</a><br>本文同步發布於 <a href="http://riverye.com/2021/10/05/Day22-%E7%94%A8-Ruby-on-Rails-%E8%99%95%E7%90%86%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E8%B3%87%E6%96%99-DB-%E8%A8%AD%E8%A8%88/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>有了前 2 篇從「臺灣證券交易所」取得 CSV 檔後，接著要把資料存入 DB，在存入前，需要先有 DB，本篇以 DB 設計為主</p>
<h2><span id="說明">說明</span></h2>
<p>
</summary>
<category term="鐵人賽" scheme="https://riverye.com/categories/%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
<category term="13th鐵人賽" scheme="https://riverye.com/tags/13th%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
</entry>
<entry>
<title>Day21 - 用 Ruby on Rails 抓臺灣證券交易所資料-除權除息計算結果表</title>
<link href="https://riverye.com/2021/10/04/Day21-%E7%94%A8-Ruby-on-Rails-%E6%8A%93%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E8%B3%87%E6%96%99-%E9%99%A4%E6%AC%8A%E9%99%A4%E6%81%AF%E8%A8%88%E7%AE%97%E7%B5%90%E6%9E%9C%E8%A1%A8/"/>
<id>https://riverye.com/2021/10/04/Day21-用-Ruby-on-Rails-抓臺灣證券交易所資料-除權除息計算結果表/</id>
<published>2021-10-03T16:00:00.000Z</published>
<updated>2022-12-21T14:31:58.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>這篇主要以抓「臺灣證券交易所」的「除權除息計算結果表」為主</p><h2><span id="取得除權除息計算結果表csv-檔">取得「除權除息計算結果表」CSV 檔</span></h2><p>目標是從臺灣證券交易所的「<a href="https://www.twse.com.tw/zh/page/trading/exchange/TWT49U.html" target="_blank" rel="noopener">除權除息計算結果表</a>」取得每日的 CSV 檔</p><p>note: 本資訊自民國92年5月5日起提供</p><p><img src="tpjTZSx.png" alt></p><h3><span id="下載的檔案內容如下">下載的檔案內容如下</span></h3><p>從上面已經知道，只提供 2003-05-05 之後的檔案</p><p><img src="10vPfTk.png" alt></p><h2><span id="實作">實作</span></h2><p>下載「<a href="https://www.twse.com.tw/zh/page/trading/exchange/TWT49U.html" target="_blank" rel="noopener">除權除息計算結果表</a>」時，資料日期可以直接從 2003-05-05 直接抓到最新一天，需留意時間範圍太大時，可能會遇到 <code>read_timeout</code> ，可選擇時間範圍別抓這麼廣，或把 <code>read_timeout</code> 時間拉長</p><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># app/features/twse/twt_49u/download.rb</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">module</span> <span class="title">Twse::Twt49u</span></span></span><br><span class="line"> <span class="class"><span class="keyword">class</span> <span class="title">Download</span></span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">include</span> Twse::Helpers</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">execute</span></span></span><br><span class="line"> current_time = Time.current</span><br><span class="line"> puts <span class="string">"<span class="subst">#{<span class="keyword">self</span>.<span class="keyword">class</span>}</span>, start_time: <span class="subst">#{current_time.to_s}</span>"</span></span><br><span class="line"> start_date = find_latest_data_date(current_time)</span><br><span class="line"> <span class="keyword">return</span> puts <span class="string">"<span class="subst">#{<span class="keyword">self</span>.<span class="keyword">class</span>}</span>, 已經是最新的資料"</span> <span class="keyword">if</span> start_date == <span class="literal">false</span></span><br><span class="line"></span><br><span class="line"> data_path = Rails.root.join(<span class="string">"data/twse/TWT49U"</span>)</span><br><span class="line"> file_path = data_path.join(<span class="string">"TWT49U_<span class="subst">#{start_date.strftime(<span class="string">"%Y%m%d"</span>)}</span>_<span class="subst">#{current_time.strftime(<span class="string">"%Y%m%d"</span>)}</span>.csv"</span>)</span><br><span class="line"> <span class="keyword">return</span> <span class="keyword">if</span> File.exists?(file_path)</span><br><span class="line"></span><br><span class="line"> FileUtils.mkdir_p(data_path) <span class="keyword">unless</span> File.directory?(data_path)</span><br><span class="line"></span><br><span class="line"> download_file(start_date, current_time, data_path, file_path)</span><br><span class="line"> upload_to_github</span><br><span class="line"> puts <span class="string">"<span class="subst">#{<span class="keyword">self</span>.<span class="keyword">class</span>}</span>, done_time:<span class="subst">#{Time.current}</span>, <span class="subst">#{(Time.current - current_time).to_s}</span> sec"</span></span><br><span class="line"> <span class="keyword">rescue</span> StandardError => e</span><br><span class="line"> puts <span class="string">"errors: <span class="subst">#{e.inspect}</span>, backtrace: <span class="subst">#{e.backtrace}</span>"</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> private</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">find_latest_data_date</span><span class="params">(current_time)</span></span></span><br><span class="line"> latest_data_date = ExStock.latest_data_date</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> latest_data_date</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span> <span class="keyword">if</span> latest_data_date == current_time</span><br><span class="line"></span><br><span class="line"> latest_data_date + <span class="number">1</span>.day</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> Date.parse(<span class="string">"2003-05-05"</span>) <span class="comment"># 僅支援抓 2003-05-05 之後的資料</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">download_file</span><span class="params">(start_date, current_time, data_path, file_path)</span></span></span><br><span class="line"> remote_file = Down::NetHttp.download(</span><br><span class="line"> <span class="string">"<span class="subst">#{BASE_URL}</span>exchangeReport/TWT49U?response=csv&strDate=<span class="subst">#{start_date.strftime(<span class="string">"%Y%m%d"</span>)}</span>&endDate=<span class="subst">#{current_time.strftime(<span class="string">"%Y%m%d"</span>)}</span>"</span>,</span><br><span class="line"> <span class="symbol">read_timeout:</span> <span class="number">120</span>,</span><br><span class="line"> )</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> remote_file.size < <span class="number">3</span></span><br><span class="line"> FileUtils.rm_rf(remote_file.path)</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> FileUtils.mv(remote_file.path, data_path.join(file_path.basename.to_s))</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><h2><span id="小結">小結</span></h2><p><code>find_latest_data_date</code> 中的 <code>ExStock</code> 為建立的 Model，這篇可以先略過，下一篇會說明 DB 的設計</p><p>有了處理「每日收盤行情」的經驗後，在處理類似的資料時，會比較快些</p><hr><p>鐵人賽文章連結：<a href="https://ithelp.ithome.com.tw/articles/10272820" target="_blank" rel="noopener">https://ithelp.ithome.com.tw/articles/10272820</a><br>medium 文章連結：<a href="https://link.medium.com/FasXk7JuTjb" target="_blank" rel="noopener">https://link.medium.com/FasXk7JuTjb</a><br>本文同步發布於 <a href="http://riverye.com/2021/10/04/Day21-%E7%94%A8-Ruby-on-Rails-%E6%8A%93%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E8%B3%87%E6%96%99-%E9%99%A4%E6%AC%8A%E9%99%A4%E6%81%AF%E8%A8%88%E7%AE%97%E7%B5%90%E6%9E%9C%E8%A1%A8/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>這篇主要以抓「臺灣證券交易所」的「除權除息計算結果表」為主</p>
<h2><span id="取得除權除息計算結果表csv-檔">取得「除權除息計算結果表」CSV 檔</span></h2>
<p>目標是從
</summary>
<category term="鐵人賽" scheme="https://riverye.com/categories/%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
<category term="13th鐵人賽" scheme="https://riverye.com/tags/13th%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
</entry>
<entry>
<title>Day20 - 用 Ruby on Rails 抓臺灣證券交易所資料-每日收盤行情</title>
<link href="https://riverye.com/2021/10/03/Day20-%E7%94%A8-Ruby-on-Rails-%E6%8A%93%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E8%B3%87%E6%96%99-%E6%AF%8F%E6%97%A5%E6%94%B6%E7%9B%A4%E8%A1%8C%E6%83%85/"/>
<id>https://riverye.com/2021/10/03/Day20-用-Ruby-on-Rails-抓臺灣證券交易所資料-每日收盤行情/</id>
<published>2021-10-02T16:00:00.000Z</published>
<updated>2022-12-21T14:30:04.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>這篇開始會有幾篇是與「臺灣證券交易所」有關，示範如何用 Ruby on Rails 來爬蟲將資料抓回來處理，並自己建立 DB，方便自己在 Local 可以測試</p><p>這部分起，不提供 sample pr or repo (個人有放 GitHub private，目前沒打算公開)</p><p>預計會示範如何抓「每日收盤行情」、「除權除息計算結果表」、設計 DB 架構、寫點簡易的技術指標選股</p><p>謎之音: 其實是原本規劃的題目，寫到膩了，想調整下內容 <s>(才不承認是拿之前自己做的 Side Project 來擠牙膏)</s></p><h2><span id="說明">說明</span></h2><p>這邊不會教股票相關的知識，我也只是一隻小菜鳥，會以技術實作層面呈現，若有寫錯或更好的寫法，歡迎不吝指教</p><h2><span id="取得每日收盤行情csv-檔">取得「每日收盤行情」CSV 檔</span></h2><p>目標是從臺灣證券交易所的「<a href="https://www.twse.com.tw/zh/page/trading/exchange/MI_INDEX.html" target="_blank" rel="noopener">每日收盤行情</a>」取得每日的 CSV 檔</p><p>note: 本資訊自民國93年2月11日起提供</p><p><img src="cjRuwPh.png" alt></p><h3><span id="下載的檔案內容如下">下載的檔案內容如下</span></h3><p>從上面已經知道，只提供 2004-02-11 之後的檔案</p><p><img src="tGEidr8.jpg" alt></p><h2><span id="實作">實作</span></h2><p>預計會用到以下 3 個 Gem，分別是 <a href="https://rubygems.org/gems/iconv" target="_blank" rel="noopener">iconv</a>、<a href="https://rubygems.org/gems/down" target="_blank" rel="noopener">down</a>、<a href="https://rubygems.org/gems/activerecord-import" target="_blank" rel="noopener">activerecord-import</a></p><p>note: Mac 的話，<code>iconv</code> 可以不用裝，最初是在 Windows 上寫的，後來變成在兩種作業系統輪流寫 XD</p><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># Gemfile</span></span><br><span class="line"></span><br><span class="line">gem <span class="string">'iconv'</span>, <span class="string">'~> 1.0'</span>, <span class="string">'>= 1.0.8'</span></span><br><span class="line">gem <span class="string">'down'</span>, <span class="string">'~> 5.2'</span>, <span class="string">'>= 5.2.2'</span></span><br><span class="line">gem <span class="string">'activerecord-import'</span>, <span class="string">'~> 1.1'</span></span><br></pre></td></tr></table></figure></p><p>由於已經知道下載、儲存 DB 會有許多方法是共用的，因此這邊直接抽 <code>helpers</code></p><p>note: 剛開始做的時候不知道，邊做邊重構，慢慢整理的，其中 <code>decode_data</code> 這方法會在存 DB 時才會用到，因此可先忽略</p><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># app/features/twse/helpers.rb</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">module</span> <span class="title">Twse</span></span></span><br><span class="line"> <span class="class"><span class="keyword">module</span> <span class="title">Helpers</span></span></span><br><span class="line"></span><br><span class="line"> BASE_URL = <span class="string">"https://www.twse.com.tw/"</span>.freeze</span><br><span class="line"></span><br><span class="line"> private</span><br><span class="line"></span><br><span class="line"> <span class="comment"># ----- download start -----</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">upload_to_github</span></span></span><br><span class="line"> need_update_github = <span class="string">`cd data/twse && git remote -v`</span></span><br><span class="line"></span><br><span class="line"> system(<span class="string">'cd data/twse && git add . && git commit -m "update data" && git push'</span>) <span class="keyword">if</span> need_update_github.present?</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="comment"># ----- download end -----</span></span><br><span class="line"></span><br><span class="line"> <span class="comment"># ----- save_to_db start -----</span></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">decode_data</span><span class="params">(file_path, is_linux)</span></span></span><br><span class="line"> <span class="keyword">if</span> is_linux</span><br><span class="line"> <span class="comment"># for windows Ubuntu</span></span><br><span class="line"> file = File.open(file_path, <span class="string">"r:UTF-8"</span>)</span><br><span class="line"> decoded_data = Iconv.iconv(<span class="string">"utf-8"</span>, <span class="string">"big-5"</span>, file.read)</span><br><span class="line"> decoded_data[<span class="number">0</span>].split(<span class="string">"\r\n"</span>)</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> <span class="comment"># for mac</span></span><br><span class="line"> file = File.open(file_path, <span class="string">"r:BIG5"</span>)</span><br><span class="line"> decoded_data = file.read</span><br><span class="line"> decoded_data.split(<span class="string">"\r\n"</span>)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="comment"># ----- save_to_db end -----</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><p>下載每日收盤行情中的「每日收盤行情(全部(不含權證、牛熊證、可展延牛熊證))」</p><p>note: 不能太密集的抓資料，會被 Bang，至於間隔幾秒怎麼知道的，trial and error 是一個方法</p><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># app/features/twse/allbut_0999/download.rb</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">module</span> <span class="title">Twse::Allbut0999</span></span></span><br><span class="line"> <span class="class"><span class="keyword">class</span> <span class="title">Download</span></span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">include</span> Twse::Helpers</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">execute</span></span></span><br><span class="line"> current_time = Time.current</span><br><span class="line"> puts <span class="string">"<span class="subst">#{<span class="keyword">self</span>.<span class="keyword">class</span>}</span>, start_time: <span class="subst">#{current_time.to_s}</span>"</span></span><br><span class="line"> data_path = Rails.root.join(<span class="string">"data/twse/ALLBUT0999"</span>)</span><br><span class="line"> start_date = find_latest_transaction_date(current_time)</span><br><span class="line"> <span class="keyword">return</span> puts <span class="string">"<span class="subst">#{<span class="keyword">self</span>.<span class="keyword">class</span>}</span>, 已經是最新的資料"</span> <span class="keyword">if</span> start_date == <span class="literal">false</span></span><br><span class="line"></span><br><span class="line"> (start_date..Date.current).each <span class="keyword">do</span> <span class="params">|date|</span></span><br><span class="line"> month_path = data_path.join(date.year.to_s, date.month.to_s)</span><br><span class="line"> file_path = month_path.join(<span class="string">"MI_INDEX_ALLBUT0999_<span class="subst">#{date.strftime(<span class="string">"%Y%m%d"</span>)}</span>.csv"</span>)</span><br><span class="line"> <span class="keyword">next</span> <span class="keyword">if</span> File.exists?(file_path) <span class="params">||</span> date.sunday?</span><br><span class="line"></span><br><span class="line"> FileUtils.mkdir_p(month_path) <span class="keyword">unless</span> File.directory?(month_path)</span><br><span class="line"></span><br><span class="line"> download_file(date, month_path)</span><br><span class="line"> sleep <span class="number">3</span> <span class="comment"># 太密集抓資料會被 bang</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> upload_to_github</span><br><span class="line"> puts <span class="string">"<span class="subst">#{<span class="keyword">self</span>.<span class="keyword">class</span>}</span>, done_time:<span class="subst">#{Time.current}</span>, <span class="subst">#{(Time.current - current_time).to_s}</span> sec"</span></span><br><span class="line"> <span class="keyword">rescue</span> StandardError => e</span><br><span class="line"> puts <span class="string">"errors: <span class="subst">#{e.inspect}</span>, backtrace: <span class="subst">#{e.backtrace}</span>"</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> private</span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">find_latest_transaction_date</span><span class="params">(current_time)</span></span></span><br><span class="line"> latest_transaction_date = DailyQuote.latest_transaction_date</span><br><span class="line"> <span class="keyword">if</span> latest_transaction_date</span><br><span class="line"> <span class="keyword">return</span> <span class="literal">false</span> <span class="keyword">if</span> latest_transaction_date == current_time</span><br><span class="line"></span><br><span class="line"> latest_transaction_date + <span class="number">1</span>.day</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> Date.parse(<span class="string">"2004-02-11"</span>) <span class="comment"># 僅支援抓 2014-02-11 之後的資料</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">download_file</span><span class="params">(date, month_path)</span></span></span><br><span class="line"> remote_file = Down.download(<span class="string">"<span class="subst">#{BASE_URL}</span>exchangeReport/MI_INDEX?response=csv&date=<span class="subst">#{date.strftime(<span class="string">"%Y%m%d"</span>)}</span>&type=ALLBUT0999"</span>)</span><br><span class="line"></span><br><span class="line"> <span class="keyword">if</span> remote_file.size.zero?</span><br><span class="line"> FileUtils.rm_rf(remote_file.path)</span><br><span class="line"> <span class="keyword">else</span></span><br><span class="line"> FileUtils.mv(remote_file.path, month_path.join(remote_file.original_filename))</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><h2><span id="小結">小結</span></h2><p>製作的過程，踩到蠻多雷的 (ex: 下載要間隔幾秒、如何處理資料...等)，可能與一開始在 Windows 上開發有關，逐一解決遇到的問題，還蠻好玩的，主要是享受開發的過程</p><p>上面有寫 <code>upload_to_github</code> 這方法，將抓下來的資料，自動上傳到 GitHub repo，若只是想單純試看看，沒有要自己建 repo 的話，這段可移除</p><p><code>find_latest_transaction_date</code> 中的 <code>DailyQuote</code> 為建立的 Model，這篇可以先略過，後面會有一篇文章說明 DB 的設計</p><p>另外也可以到「<a href="https://openapi.twse.com.tw/" target="_blank" rel="noopener">臺灣證券交易所 OpenAPI</a>」打 API 取資料，這邊就不多闡述了</p><p>下一篇會示範如何抓「除權除息計算結果表」的資料</p><hr><p>鐵人賽文章連結：<a href="https://ithelp.ithome.com.tw/articles/10272778" target="_blank" rel="noopener">https://ithelp.ithome.com.tw/articles/10272778</a><br>medium 文章連結：<a href="https://link.medium.com/DtL2qRIuTjb" target="_blank" rel="noopener">https://link.medium.com/DtL2qRIuTjb</a><br>本文同步發布於 <a href="http://riverye.com/2021/10/03/Day20-%E7%94%A8-Ruby-on-Rails-%E6%8A%93%E8%87%BA%E7%81%A3%E8%AD%89%E5%88%B8%E4%BA%A4%E6%98%93%E6%89%80%E8%B3%87%E6%96%99-%E6%AF%8F%E6%97%A5%E6%94%B6%E7%9B%A4%E8%A1%8C%E6%83%85/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>這篇開始會有幾篇是與「臺灣證券交易所」有關，示範如何用 Ruby on Rails 來爬蟲將資料抓回來處理，並自己建立 DB，方便自己在 Local 可以測試</p>
<p>這部分起，不提供 sample p
</summary>
<category term="鐵人賽" scheme="https://riverye.com/categories/%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
<category term="13th鐵人賽" scheme="https://riverye.com/tags/13th%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
</entry>
<entry>
<title>禱告事項</title>
<link href="https://riverye.com/2021/10/03/fasted-and-prayed/"/>
<id>https://riverye.com/2021/10/03/fasted-and-prayed/</id>
<published>2021-10-02T16:00:00.000Z</published>
<updated>2022-12-21T14:31:42.000Z</updated>
<content type="html"><![CDATA[<div id="hexo-blog-encrypt" data-wpm="Oh, this is an invalid password. Check and try again, please." data-whm="OOPS, these decrypted content may changed, but you can still have a look."> <div class="hbe-input-container"> <input type="password" id="hbePass" placeholder="猜對就給你看"> <label for="hbePass">猜對就給你看</label> <div class="bottom-line"></div> </div> <script id="hbeData" type="hbeData" data-hmacdigest="68d7c04418e020b1fb2921db711481329615d7adbb20a7431fc24057efaeb866">6a160f69659dbd354f4bdd84f335229b5fa4d4a4035d06d8b416fcf72ce591554467cb27e1d66077effb5390c0a944ae300aba9d2082eb02ee76e3bdded90d469949c44e69a1c7413a298e1963cef2b2fa39883373f8ec88e9f6159d291017cd740e687c2b123414a602f9675d66d608f1592bbd5eec8c88ee30ab6f61ec4c9167be80e7b5099cbbda15c4b140e4fc25cb5e68d5428dd47e661997135549f9c00421c397c9295d256a0e5b90c04cd6231ee4b840757738691405bb67a4b9c8d8dacb95895a16452102def7de85d9f488cf676e76aa44f110e8381bf641df45e0c8205351129e99c69d2510d7702c76c61555d96826713a4651a65a4f9852a7de39f3462bc2a72f9c653b3a2c8c7cc05a8613a6d87802f12f112e8f7a327086c4e4b5e088028a70c8abaf4a47b9c3c52e892bb5eafeaa4fa6d96aa6a0a53d1b20c49dfdf30f7b98a9e2936982c3adb94ef7b685c7571aa126a1865bcfd4f47eb4abfa4b66727627e41d18136c0b8ecbab351796ed7f7c0a20f80305d00b3fbffa86471ab5c584cc2b6881424ed548136d3b0557f102fd6aa9047cfd4d1ab1441bd072a6df1d0ad785f56a6d4c371128b5bc6c6eb641cbd33382508116cf0e2a89ec8bc60d91430b4793eabbec8d5d027d9940580f26d6fbeab705d33f922dc09620f991aa53f90b82f32e01395e6a86c85222caf164766d98cf6391a1b56ad47a481bb2bdcf449562384e090a8b20f68bd93eef686b02f27a6a289b23cd6604244e906a81918fcf16cf7840b4f08a7e063803595a40bcda6a434784f139ff26b99aee18bbde6cdb9d0ef3e9f86bab87cc40b75fdd9c0384c82cb8a75c087eab75b82f1b1ebb46cec0faa4220c42754635225d7449f2db6acda20b19bf7ca3d1ee6a0db29810bf1044a18f4860a252388e8fd2870f5c3357de34357d1543e31d600eb846614463c1974c358669d041b2c2d350aad137829e8bb63f7acddd5a55042e6eecdca7b10603262b2b7525ad365dc15353f435cf5bb63f3d49d58a5414e531a08f53abfc97156dd2e0b2e7bc83670855fda76efe05d5f052826b7e98301252a4808fa621e26d7e231efe94a296a195e6e29589439ac459585dbbcd353162041f816e2a58428f883b324a7d6c3aad363403082b0c8b11b34aada2198da2f7ccbe9ecb52776e42fe052a559a9fc921f166cb411db7ee5dbdc864553fa2a8877b32b46ba8aece67d4a36811ceacdf06d6a0d80f52057d018c5d9e6cd84f4f920e02266f52fe91d9f25818951bd8b0355a8679e83efda6f52ef7ff45469a816ab2d70f1436d36676b0d033b68cb0c23e7f01d901845fb9398ed3dc2fd5db3d95dd1ac696d32fece77c57b028223798c27703958e74ddc33346d5fc5b4b2782293f1793fce837b9a0275c34883edae9600aa246b3d0b8dcfcf1b77ddcd565af77f714b479cfe02ffb1fd3c078ffd1696349bbd85bfe23ec034a9929e53a0def3393e00259c6f5b59e631bf6fbac602e64a54ae9d85f2fdafd068fcfd49ca9d8bf3e82ab9cd179f83d41f88dbb2c93fbb1a11f3bc68b9065f96342a9b90456c2363b5e9e666d3fefa96276a9a21134b9cded2d42d50c583049bb1ae5d1753882178ca87adef56bdba92605a2c72c45e10153cf762e680c0d0e72002e840b0794042234c00d098b3e91d2a334b3384e29503138b16c38976c17107290d28f4cdcf86b4e657fd2c3ebe2984e1321cea57f68a14fce13e96a7d64da7758e59fcadd18afbce4cbf648fbbe226c4c0e6323fd04f974473eb072aee54294b7585c39542c2caf1a87961aa963e21b6c9088993654435f735d3620fbd7bf728d2d734db14f4443f443230a661b48fca611fe41435dfb31c5c4600a784e5c1a75bcb91e1ae7b379c4b171d85104d7e8762506cdeab8d00bb293d5b6ea2d733bb9e79f55738f51569f86a2c4ff46c292077daa634565a45e4e5e58f86e710a91cf39e521c6cdd1b16e78ece895064cc241ed31dc5f9f42604667e7ed048f158c9a7f9005192cdbca1d96b53b6ba2f6d4784e898fe47e833ddb02a397dc77a5cd50d43f28584e73fd82b47f3d9e3f459c9e19e950ae70e25b2812bea4519277034df4bf58ca85d459e17d308e0868b5a2f648efc07f42006e9c89d7140698b1e36cfb52b59bf1c910a82bca3b9268f087db4e212e8ccef793dc505698cdbcdcfdec9e4f008ce3465cba3729e5e451647799681d712018dbe8943bcd8c1d59d5c12c5b14b36b77dc626f64466b373da330f3380d4d8ed32616ef7ab84cec6dbbd373124624dbca1d8d54fd0e34428312e1d7998af9c6ca70724591904ba4be69d827cb466881791a100032d48ccbe908362bd7ac4a8c6306a4d4613b1bb144c12d7495dbcb4b0acc61fef5c2bad648ed1bb980826696f4be7852c48869e4674820cfe1ed08f722b1e5020115011cc10b554ec3da78d5f9f40567076efc55af6d24c04a60fc6e87d34227901ba29b645661f6732f376c60b5402826e04a3e1ae30aacf58a0473691dd200fdf9843251e095646f57f73d2308933a20f72d3cfa5cc43e658ec101aa474f08623cc1bed3a9b786da5a33f6ebeb24934d413478e8e56e0ceef3d0af9f3ace1fae688788479e26108101e12ac8a1834bb3e47e5b807f612845b7614c20bfb8ef253a21559b8335523f348f4345b0f44e292a8fc03adbb172c5a01ae5285cf25f32027b85e79e24efaae4cfb80ab1cb71f1ea1e8e5839034c6085bdb410195c14bbf2a66822fa85bdcf22218d5ce92c6056b20f39d04d8b20ebf2363fd9b8bd92c34d80ea68748445624bfc94e6b33098d1c7097a75e72eb2a4e988a9e386d64a30dd23825c6b6239e4c808d1b3fde732bc9f94295deada3db6c17b834d6f5433fbd53d229fd4fe97f2ff78aea70aa3d6a4c29dc2929266d88d542f082903712fc077ddafc1aa2fc693f0f2f3bf86da35a8c43c1562d98b621e72617a2481074b76defef01c2b466e6e4b59388627ed6a8b618b6c19e573f70e8bc5041c0b14acd4e1bbb03d0327053e5ecc68bfde03487102e34ca91a5c642277e60ba3db73a84f17daad12c2a33504596d52566cf4bc48975da16a53aac1b96e914ec71d5b99b79ebefc66d030ec25d1d6a665835c87ffc5c4738730a512a5e1bd0a7a624a3be94ed5a16569a573fb4e75e4608d8ceeafd7ae5d750b78c1f01e7ac5271b3287970ccd645757209bb54447543ae77dd999ae2d84d0c62f653e445f87870c804056b47b6c9b9bbe4206580187be1213d5e5402ef9f68742cbca29cbe724863f202e06b304b50d45374397adbdedb03e10356f19fa091e5c5e170182a8b143038dee2af7ac67563caa3e1021750dfc4f00c8e257f8598f11d19352a56ea59d461428e910ce72321b00fc3c118b348af29e7edadd00a7b51deac215fecc1e95f23f89b9133ca3ebb4478606a22e23446febe4add2ebb6e0ad8c672261a6308a4d014e2054839d349fce38f6e83947efeb9b7713d7438c85fef3722acb56430a9b76870466db6ab8cad5deed81076169836c2ce8f8dfbb75bacc7956642be149ce2967422b605ee5510c8d1c082edc5334c375a1961a28560065f0cabf8164f9429ac4fe80844034075ddeb5d724fb</script></div><script src="/lib/blog-encrypt.js"></script><link href="/css/blog-encrypt.css" rel="stylesheet" type="text/css">]]></content>
<summary type="html">
輸入密碼才看的到喔
</summary>
<category term="禱告" scheme="https://riverye.com/tags/%E7%A6%B1%E5%91%8A/"/>
</entry>
<entry>
<title>Day19 - 匯入 excel-測試篇</title>
<link href="https://riverye.com/2021/10/02/Day19-%E5%8C%AF%E5%85%A5-excel-%E6%B8%AC%E8%A9%A6%E7%AF%87/"/>
<id>https://riverye.com/2021/10/02/Day19-匯入-excel-測試篇/</id>
<published>2021-10-01T16:00:00.000Z</published>
<updated>2022-09-23T08:45:10.000Z</updated>
<content type="html"><![CDATA[<h2><span id="前言">前言</span></h2><p>繼<a href="https://ithelp.ithome.com.tw/articles/10272591" target="_blank" rel="noopener">上篇</a>匯入 Excel 實作，這篇以撰寫測試為主</p><h2><span id="實作">實作</span></h2><p>測試的寫法有蠻多種，這邊以其中一種為例，可參考此 <a href="https://github.com/River-Ye/ironman_13th_2021/pull/9" target="_blank" rel="noopener">pr</a></p><p><figure class="highlight ruby"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># spec/services/shops_excel/parser_spec.rb</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">require</span> <span class="string">'rails_helper'</span></span><br><span class="line"></span><br><span class="line">RSpec.describe ShopsExcel::Parser <span class="keyword">do</span></span><br><span class="line"> describe <span class="string">'execute'</span> <span class="keyword">do</span></span><br><span class="line"> let(<span class="symbol">:excel_file</span>) { Tempfile.new(<span class="string">%w[shops_import .xlsx]</span>) }</span><br><span class="line"> let(<span class="symbol">:excel</span>) <span class="keyword">do</span></span><br><span class="line"> file = excel_file</span><br><span class="line"> file.binmode</span><br><span class="line"></span><br><span class="line"> xlsx = Axlsx::Package.new</span><br><span class="line"> workbook = xlsx.workbook</span><br><span class="line"> workbook.add_worksheet(<span class="symbol">name:</span> <span class="string">'商家清單'</span>) <span class="keyword">do</span> <span class="params">|sheet|</span></span><br><span class="line"> sheet.add_row(excel_titles)</span><br><span class="line"> sheet.add_row(content_array1)</span><br><span class="line"> sheet.add_row(content_array2)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> xlsx.use_shared_strings = <span class="literal">true</span></span><br><span class="line"> xlsx.serialize(file)</span><br><span class="line"></span><br><span class="line"> file.rewind</span><br><span class="line"> file</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> let(<span class="symbol">:excel_titles</span>) { ShopsExcel::Generator::TITLES }</span><br><span class="line"> let(<span class="symbol">:content_array1</span>) { [<span class="string">'shop_name_1'</span>, <span class="string">'river@riverye.com'</span>, <span class="string">'hello'</span>] }</span><br><span class="line"> let(<span class="symbol">:content_array2</span>) { [<span class="string">'shop_name_2'</span>, <span class="string">'test@riverye.com'</span>, <span class="string">'world'</span>] }</span><br><span class="line"></span><br><span class="line"> subject { described_class.new.execute(excel) }</span><br><span class="line"></span><br><span class="line"> context <span class="string">'create shops successfully'</span> <span class="keyword">do</span></span><br><span class="line"> it <span class="string">'data correctly'</span> <span class="keyword">do</span></span><br><span class="line"> expect { subject }.to change { Shop.count }.by(<span class="number">2</span>)</span><br><span class="line"> [content_array1, content_array2].each <span class="keyword">do</span> <span class="params">|content_array|</span></span><br><span class="line"> shop1 = Shop.find_by(<span class="symbol">name:</span> content_array[<span class="number">0</span>])</span><br><span class="line"> expect(shop1).to have_attributes(</span><br><span class="line"> <span class="symbol">name:</span> content_array[<span class="number">0</span>],</span><br><span class="line"> <span class="symbol">email:</span> content_array[<span class="number">1</span>],</span><br><span class="line"> <span class="symbol">note:</span> content_array[<span class="number">2</span>],</span><br><span class="line"> )</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> context <span class="string">'when return error'</span> <span class="keyword">do</span></span><br><span class="line"> let(<span class="symbol">:logger</span>) { double }</span><br><span class="line"></span><br><span class="line"> before <span class="keyword">do</span></span><br><span class="line"> allow(Rails).to receive(<span class="symbol">:logger</span>).and_return(logger)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> <span class="function"><span class="keyword">def</span> <span class="title">expect_error_message</span><span class="params">(error_message)</span></span></span><br><span class="line"> expect(logger).to receive(<span class="symbol">:error</span>).with(<span class="regexp">/\[ShopsExcel::Parser Error\] <span class="subst">#{error_message}</span>/</span>)</span><br><span class="line"> expect { subject }.to change { Shop.count }.by(<span class="number">0</span>)</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> context <span class="string">'when title is empty'</span> <span class="keyword">do</span></span><br><span class="line"> let(<span class="symbol">:excel_titles</span>) { [<span class="literal">nil</span>, <span class="literal">nil</span>, <span class="literal">nil</span>] }</span><br><span class="line"> let(<span class="symbol">:error_message</span>) { <span class="string">'輸入資料有誤，比對 Excel 標頭與預期不同'</span> }</span><br><span class="line"></span><br><span class="line"> it { expect_error_message(error_message) }</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> context <span class="string">'when data is empty'</span> <span class="keyword">do</span></span><br><span class="line"> let(<span class="symbol">:content_array1</span>) { [<span class="literal">nil</span>, <span class="literal">nil</span>, <span class="literal">nil</span>] }</span><br><span class="line"> let(<span class="symbol">:content_array2</span>) { content_array1 }</span><br><span class="line"> let(<span class="symbol">:error_message</span>) { <span class="string">'無資料'</span> }</span><br><span class="line"></span><br><span class="line"> it { expect_error_message(error_message) }</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> context <span class="string">'when data is duplicate'</span> <span class="keyword">do</span></span><br><span class="line"> let(<span class="symbol">:content_array2</span>) { content_array1 }</span><br><span class="line"> let(<span class="symbol">:error_message</span>) { <span class="string">'有重複的商家名稱，請檢查'</span> }</span><br><span class="line"></span><br><span class="line"> it { expect_error_message(error_message) }</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"></span><br><span class="line"> context <span class="string">'when shop exists'</span> <span class="keyword">do</span></span><br><span class="line"> let(<span class="symbol">:shop</span>) { create(<span class="symbol">:shop</span>) }</span><br><span class="line"> let(<span class="symbol">:content_array1</span>) { [shop.name, shop.email, shop.note] }</span><br><span class="line"> let(<span class="symbol">:content_array2</span>) { [<span class="string">'shop_name_2'</span>, <span class="string">'test@riverye.com'</span>, <span class="string">'world'</span>] }</span><br><span class="line"> let(<span class="symbol">:error_message</span>) { <span class="string">"有 1 筆已建立過: <span class="subst">#{shop.name}</span>"</span> }</span><br><span class="line"></span><br><span class="line"> it { expect_error_message(error_message) }</span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"> <span class="keyword">end</span></span><br><span class="line"><span class="keyword">end</span></span><br></pre></td></tr></table></figure></p><h2><span id="小結">小結</span></h2><p>這篇其實是與<a href="https://ithelp.ithome.com.tw/articles/10272591" target="_blank" rel="noopener">上篇</a>一起先寫好 code ，接著才開始寫文章，在寫測試過程中，有發現原本寫的方法要微調，這也是撰寫測試的好處，能發現一些沒留意到的眉眉角角 <s>(說這麼多，就是擠牙膏嘛)</s></p><p>若有更好的寫法，歡迎留言和我說~</p><h2><span id="參考資料">參考資料</span></h2><ol><li><a href="https://github.com/roo-rb/roo" target="_blank" rel="noopener">Roo GitHub</a></li></ol><hr><p>鐵人賽文章連結：<a href="https://ithelp.ithome.com.tw/articles/10272599" target="_blank" rel="noopener">https://ithelp.ithome.com.tw/articles/10272599</a><br>medium 文章連結：<a href="https://link.medium.com/sYWyCHbyRjb" target="_blank" rel="noopener">https://link.medium.com/sYWyCHbyRjb</a><br>本文同步發布於 <a href="http://riverye.com/2021/10/02/Day19-%E5%8C%AF%E5%85%A5-excel-%E6%B8%AC%E8%A9%A6%E7%AF%87/">小菜的 Blog</a> <a href="https://riverye.com/">https://riverye.com/</a></p><p>備註：之後文章修改更新，以個人部落格為主</p>]]></content>
<summary type="html">
<h2><span id="前言">前言</span></h2>
<p>繼<a href="https://ithelp.ithome.com.tw/articles/10272591" target="_blank" rel="noopener">上篇</a>匯入 Excel
</summary>
<category term="鐵人賽" scheme="https://riverye.com/categories/%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
<category term="13th鐵人賽" scheme="https://riverye.com/tags/13th%E9%90%B5%E4%BA%BA%E8%B3%BD/"/>
</entry>
</feed>