眠気.jl

投稿=不定期

Juliaでスクレイピング 1/m

JuliaでGETとかしたい(with HTTP.jl)

動機

元ネタのJulia本の方だと、英語のwikiなので,日本語もやってみたくなった

目標

JuliaでJuliaの日本語wikiを読む
つまり、ここに行って、中身取得してくる

作業環境

今回の環境(前回同様)

julia> versioninfo()
Julia Version 1.1.0
...

やってみる

julia>
(v1.1) pkg> add HTTP
julia> edit("first_http.jl")

次の様に編集する

using HTTP

Julia言語のページ☆😺☆="https://ja.wikipedia.org/wiki/Julia_(%E3%83%97%E3%83%AD%E3%82%B0%E3%83%A9%E3%83%9F%E3%83%B3%E3%82%B0%E8%A8%80%E8%AA%9E)"
ja_wiki = HTTP.get(Julia言語のページ☆😺☆)
println(ja_wiki)
println(typeof(ja_wiki))
println(fieldnames(typeof(ja_wiki)))

実行してみると,

shell> julia http.jl
HTTP.Messages.Response:
"""
HTTP/1.1 200 OK
Date: Tue, 26 Feb 2019 10:48:28 GMT
Content-Type: text/html; charset=UTF-8
...(ここすごい長いので略)
<!DOCTYPE html>
<html class="client-nojs" lang="ja" dir="ltr">
<head>
<meta charset="UTF-8"/>
<title>Julia (プログラミング言語) - Wikipedia</title>
<script>document.documentElement.className = document.documentElement.className.replace( /(^|\s)client-nojs(\s|$)/, "$1client-js$2" );</script>
<script>(window.RLQ=window.RLQ||[]).push(function(){mw.config.set({"wgCanonicalNamespace":"","wgCanonicalSpecialPageName":false,"wgNamespaceNumber":0,"wgPageName":"Julia_(プログラミング言語)","wgTitle":"Julia (プログラミング言語)","wgCurRevisionId":71653479,"wgRevisionId":71653479,"wgArticleId":2911591,"wgIsArticle":true,"wgIsRedirect":false,"wgAction":"view","wgUserName":null,"wgUserGroups":["*"],"wgCategories":["ウィキペディアとウィキデータで異なる公式ウェブサイト","FLOSS関連のスタブ項目","オブジェクト指向言語","数値線形代数","並列コンピューティング","プログラミング言語","Linux向け数値解析ソフ
77561-byte body
"""
HTTP.Messages.Response
(:version, :status, :headers, :body, :request)

と帰ってくる
HTTP.Messages.Responseという形で帰ってくることがわかる
(fieldnames=filedの名前をタプルでとってくる関数)
body以外を順番に見る,次のように書き換えてから実行

using HTTP

Julia言語のページ☆😺☆="https://ja.wikipedia.org/wiki/Julia_(%E3%83%97%E3%83%AD%E3%82%B0%E3%83%A9%E3%83%9F%E3%83%B3%E3%82%B0%E8%A8%80%E8%AA%9E)"
ja_wiki = HTTP.get(Julia言語のページ☆😺☆)
println(ja_wiki)

println(ja_wiki.version)
println(ja_wiki.status)
println(ja_wiki.headers)
#println(ja_wiki.body)
println(ja_wiki.request)

すると

HTTP.Messages.Response:
"""
HTTP/1.1 200 OK
Date: Tue, 26 Feb 2019 11:16:51 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 77561
Connection: keep-alive
Server: mw1268.eqiad.wmnet
X-Content-Type-Options: nosniff
P3P: CP="This is not a P3P policy! See https://ja.wikipedia.org/wiki/%E7%89%B9%E5%88%A5:CentralAutoLogin/P3P for more info."
X-Powered-By: HHVM/3.18.6-dev
Content-language: ja
Last-Modified: Sun, 24 Feb 2019 18:56:33 GMT
Backend-Timing: D=116477 t=1551045681198724
Vary: Accept-Encoding,Cookie,Authorization,X-Seven
X-Varnish: 491791512 489799321, 1005959020, 232551754, 72497187 690231799
Via: 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1)
Age: 51391
X-Cache: cp1085 hit/9, cp2012 pass, cp5007 miss, cp5009 hit/35
X-Cache-Status: hit-front
Server-Timing: cache;desc="hit-front"
Strict-Transport-Security: max-age=106384710; includeSubDomains; preload
Set-Cookie: WMF-Last-Access=26-Feb-2019;Path=/;HttpOnly;secure;Expires=Sat, 30 Mar 2019 00:00:00 GMT
Set-Cookie: WMF-Last-Access-Global=26-Feb-2019;Path=/;Domain=.wikipedia.org;HttpOnly;secure;Expires=Sat, 30 Mar 2019 00:00:00 GMT
X-Analytics: ns=0;page_id=2911591;https=1;nocookies=1
X-Client-IP: 111.239.190.1
Cache-Control: private, s-maxage=0, max-age=0, must-revalidate
Set-Cookie: GeoIP=JP:13:Tokyo:35.70:139.67:v4; Path=/; secure; Domain=.wikipedia.org
Accept-Ranges: bytes

<!DOCTYPE html>
<html class="client-nojs" lang="ja" dir="ltr">
<head>
<meta charset="UTF-8"/>
<title>Julia (プログラミング言語) - Wikipedia</title>
<script>document.documentElement.className = document.documentElement.className.replace( /(^|\s)client-nojs(\s|$)/, "$1client-js$2" );</script>
<script>(window.RLQ=window.RLQ||[]).push(function(){mw.config.set({"wgCanonicalNamespace":"","wgCanonicalSpecialPageName":false,"wgNamespaceNumber":0,"wgPageName":"Julia_(プログラミング言語)","wgTitle":"Julia (プログラミング言語)","wgCurRevisionId":71653479,"wgRevisionId":71653479,"wgArticleId":2911591,"wgIsArticle":true,"wgIsRedirect":false,"wgAction":"view","wgUserName":null,"wgUserGroups":["*"],"wgCategories":["ウィキペディアとウィキデータで異なる公式ウェブサイト","FLOSS関連のスタブ項目","オブジェクト指向言語","数値線形代数","並列コンピューティング","プログラミング言語","Linux向け数値解析ソフ�
⋮
77561-byte body
"""
1.1.0
200
Pair{SubString{String},SubString{String}}["Date"=>"Tue, 26 Feb 2019 11:16:51 GMT", "Content-Type"=>"text/html; charset=UTF-8", "Content-Length"=>"77561", "Connection"=>"keep-alive", "Server"=>"mw1268.eqiad.wmnet", "X-Content-Type-Options"=>"nosniff", "P3P"=>"CP=\"This is not a P3P policy! See https://ja.wikipedia.org/wiki/%E7%89%B9%E5%88%A5:CentralAutoLogin/P3P for more info.\"", "X-Powered-By"=>"HHVM/3.18.6-dev", "Content-language"=>"ja", "Last-Modified"=>"Sun, 24 Feb 2019 18:56:33 GMT", "Backend-Timing"=>"D=116477 t=1551045681198724", "Vary"=>"Accept-Encoding,Cookie,Authorization,X-Seven", "X-Varnish"=>"491791512 489799321, 1005959020, 232551754, 72497187 690231799", "Via"=>"1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1)", "Age"=>"51391", "X-Cache"=>"cp1085 hit/9, cp2012 pass, cp5007 miss, cp5009 hit/35", "X-Cache-Status"=>"hit-front", "Server-Timing"=>"cache;desc=\"hit-front\"", "Strict-Transport-Security"=>"max-age=106384710; includeSubDomains; preload", "Set-Cookie"=>"WMF-Last-Access=26-Feb-2019;Path=/;HttpOnly;secure;Expires=Sat, 30 Mar 2019 00:00:00 GMT", "Set-Cookie"=>"WMF-Last-Access-Global=26-Feb-2019;Path=/;Domain=.wikipedia.org;HttpOnly;secure;Expires=Sat, 30 Mar 2019 00:00:00 GMT", "X-Analytics"=>"ns=0;page_id=2911591;https=1;nocookies=1", "X-Client-IP"=>"111.239.190.1", "Cache-Control"=>"private, s-maxage=0, max-age=0, must-revalidate", "Set-Cookie"=>"GeoIP=JP:13:Tokyo:35.70:139.67:v4; Path=/; secure; Domain=.wikipedia.org", "Accept-Ranges"=>"bytes"]
HTTP.Messages.Request:
"""
GET /wiki/Julia_(%E3%83%97%E3%83%AD%E3%82%B0%E3%83%A9%E3%83%9F%E3%83%B3%E3%82%B0%E8%A8%80%E8%AA%9E) HTTP/1.1
Host: ja.wikipedia.org
Content-Length: 0

"""

と帰ってくる,bodyを見てみると

HTTP.Messages.Response:
"""
HTTP/1.1 200 OK
Date: Tue, 26 Feb 2019 11:20:59 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 77561
Connection: keep-alive
Server: mw1268.eqiad.wmnet
X-Content-Type-Options: nosniff
P3P: CP="This is not a P3P policy! See https://ja.wikipedia.org/wiki/%E7%89%B9%E5%88%A5:CentralAutoLogin/P3P for more info."
X-Powered-By: HHVM/3.18.6-dev
Content-language: ja
Last-Modified: Sun, 24 Feb 2019 18:56:33 GMT
Backend-Timing: D=116477 t=1551045681198724
Vary: Accept-Encoding,Cookie,Authorization,X-Seven
X-Varnish: 491791512 489799321, 1005959020, 232551754, 84021737 690231799
Via: 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1)
Age: 51639
X-Cache: cp1085 hit/9, cp2012 pass, cp5007 miss, cp5009 hit/37
X-Cache-Status: hit-front
Server-Timing: cache;desc="hit-front"
Strict-Transport-Security: max-age=106384710; includeSubDomains; preload
Set-Cookie: WMF-Last-Access=26-Feb-2019;Path=/;HttpOnly;secure;Expires=Sat, 30 Mar 2019 00:00:00 GMT
Set-Cookie: WMF-Last-Access-Global=26-Feb-2019;Path=/;Domain=.wikipedia.org;HttpOnly;secure;Expires=Sat, 30 Mar 2019 00:00:00 GMT
X-Analytics: ns=0;page_id=2911591;https=1;nocookies=1
X-Client-IP: 111.239.190.1
Cache-Control: private, s-maxage=0, max-age=0, must-revalidate
Set-Cookie: GeoIP=JP:13:Tokyo:35.70:139.67:v4; Path=/; secure; Domain=.wikipedia.org
Accept-Ranges: bytes

<!DOCTYPE html>
<html class="client-nojs" lang="ja" dir="ltr">
<head>
<meta charset="UTF-8"/>
<title>Julia (プログラミング言語) - Wikipedia</title>
<script>document.documentElement.className = document.documentElement.className.replace( /(^|\s)client-nojs(\s|$)/, "$1client-js$2" );</script>
<script>(window.RLQ=window.RLQ||[]).push(function(){mw.config.set({"wgCanonicalNamespace":"","wgCanonicalSpecialPageName":false,"wgNamespaceNumber":0,"wgPageName":"Julia_(プログラミング言語)","wgTitle":"Julia (プログラミング言語)","wgCurRevisionId":71653479,"wgRevisionId":71653479,"wgArticleId":2911591,"wgIsArticle":true,"wgIsRedirect":false,"wgAction":"view","wgUserName":null,"wgUserGroups":["*"],"wgCategories":["ウィキペディアとウィキデータで異なる公式ウェブサイト","FLOSS関連のスタブ項目","オブジェクト指向言語","数値線形代数","並列コンピューティング","プログラミング言語","Linux向け数値解析ソフ�
⋮
77561-byte body
"""
UInt8[0x3c, 0x21, 0x44, 0x4f, 0x43, 0x54, 0x59, 0x50, 0x45, 0x20, 0x68, 0x74, 0x6d, 0x6c, 0x3e, 0x0a, 0x3c, 0x68, 0x74, 0x6d, 0x6c, 0x20, 0x63, 0x6c, 0x61, 0x73, 0x73, 0x3d, 0x22, 0x63, 0x6c, 0x69, 0x65, 0x6e, 0x74, 0x2d, 0x6e, 0x6f, 0x6a, 0x73, 0x22, 0x20, 0x6c, 0x61, 0x6e, 0x67, 0x3d, 0x22, 0x6a, 0x61, 0x22, 0x20, 0x64, 0x69, 0x72, 0x3d, 0x22, 0x6c, 0x74, 0x72, 0x22, 0x3e, 0x0a, 0x3c, 0x68, 0x65, 0x61, 0x64, 0x3e, 0x0a, 0x3c, 0x6d, 0x65, 0x74, 0x61, 0x20, 0x63, 0x68, 0x61, 0x72, 0x73, 0x65, 0x74, 0x3d, 0x22, 0x55, 0x54, 0x46, 0x2d, 0x38, 0x22, 0x2f, 0x3e, 0x0a, 0x3c, 0x74, 0x69, 0x74, 0x6c, 0x65, 0x3e, 0x4a, 0x75, 0x6c, 0x69, 0x61, 0x20, 0x28, 0xe3, 0x83, 0x97, 0xe3, 0x83, 0xad, 0xe3, 0x82, 0xb0, 0xe3, 0x83, 0xa9, 0xe3, 0x83, 0x9f, 0xe3, 0x83, 0xb3, 0xe3, 0x82, 0xb0, 0xe8, 0xa8, 0x80, 0xe8, 0xaa, 0x9e, 0x29, 0x20, 0x2d, 0x20, 0x57, 0x69, 0x6b, 0x69, 0x70, 0x65, 0x64, 0x69, 0x61, 0x3c, 0x2f, 0x74, 0x69, 0x74, 0x6c, 0x65, 0x3e, 0x0a, 0x3c, 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x3e, 0x64, 0x6f, 0x63, 0x75, 0x6d, 0x65, 0x6e, 0x74, 0x2e, 0x64, 0x6f, 0x63, 0x75, 0x6d, 0x65, 0x6e, 0x74, 0x45, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x2e, 0x63, 0x6c, 0x61, 0x73, 0x73, 0x4e, 0x61, 0x6d, 0x65, 0x20, 0x3d, 0x20, 0x64, 0x6f, 0x63, 0x75, 0x6d, 0x65, 0x6e, 0x74, 0x2e, 0x64, 0x6f, 0x63, 0x75, 0x6d, 0x65, 0x6e, 0x74, 0x45, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x2e, 0x63, 0x6c, 0x61, 0x73, 0x73, 0x4e, 0x61, 0x6d, 0x65, 0x2e, 0x72, 0x65, 0x70, 0x6c, 0x61, 0x63, 0x65, 0x28, 0x20, 0x2f, 0x28, 0x5e, 0x7c, 0x5c, 0x73, 0x29, 0x63, 0x6c, 0x69, 0x65, 0x6e, 0x74, 0x2d, 0x6e, 0x6f, 0x6a, 0x73, 0x28, 0x5c, 0x73, 0x7c, 0x24, 0x29, 0x2f, 0x2c, 0x20, 0x22, 0x24, 0x31, 0x63, 0x6c, 0x69, 0x65, 0x6e, 0x74, 0x2d, 0x6a, 0x73, 0x24, 0x32, 0x22, 0x20, 0x29, 0x3b, 0x3c, 0x2f, 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x3e, 0x0a, 0x3c, 0x73, 0x63, 0x72, 0x69, 0x70, 0x74, 0x3e, 0x28, 0x77, 0x69, 0x6e, 0x64, 0x6f, 0x77, 0x2e, 0x52, 0x4c, 0x51, 0x3d, 0x77, 0x69, 0x6e, 0x64, 0x6f, 0x77, 0x2e, 0x52, 0x4c, 0x51, 0x7c, 0x7c, 0x5b, 0x5d, 0x29, 0x2e, 0x70, 0x75, 0x73, 0x68, 0x28, 0x66, 0x75, 0x6e, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x28, 0x29, 0x7b, 0x6d, 0x77, 0x2e, 0x63, 0x6f, 0x6e, 0x66, 0x69, 0x67, 0x2e, 0x73, 0x65, 0x74, 0x28, 0x7b, 0x22, 0x77, 0x67, 0x43, 0x61, 0x6e, 0x6f, 0x6e, 0x69, 0x63, 0x61, 0x6c, 0x4e, 0x61, 0x6d, 0x65, 0x73, 0x70, 0x61, 0x63, 0x65, 0x22, 0x3a, 0x22, 0x22, 0x2c, 0x22, 0x77, 0x67, 0x43, 0x61, 0x6e, 0x6f, 0x6e, 0x69, 0x63, 0x61, 0x6c, 0x53, 0x70, 0x65, 0x63, 0x69, 0x61, 0x6c, 0x50, 0x61, 0x67, 0x65, 0x4e, 0x61, 0x6d, 0x65, 0x22, 0x3a, 0x66, 0x61, 0x6c, 0x73, 0x65, 0x2c, 0x22, 0x77, 0x67, 0x4e, 0x61, 0x6d, 0x65, 0x73, 0x70, 0x61, 0x63, 0x65, 0x4e, 0x75, 0x6d, 0x62, 0x65, 0x72, 0x22, 0x3a, 0x30, 0x2c, 0x22, 0x77, 0x67, 0x50, 0x61, 0x67, 0x65, 0x4e, 0x61, 0x6d, 0x65, 0x22, 0x3a, 0x22, 0x4a, 0x75, 0x6c, 0x69, 0x61, 0x5f, 0x28, 0xe3, 0x83, 0x97, 0xe3, 0x83, 0xad, 0xe3, 0x82, 0xb0, 0xe3, 0x83, 0xa9, 0xe3, 0x83, 0x9f, 0xe3, 0x83, 0xb3, 0xe3, 0x82, 0xb0, 0xe8, 0xa8, 0x80, 0xe8, 0xaa, 0x9e, 0x29, 0x22, 0x2c, 0x22, 0x77, 0x67, 0x54, 0x69, 0x74, 0x6c, 0x65, 0x22, 0x3a, 0x22, 0x4a, 0x75, 0x6c, 0x69, 0x61, 0x20, 0x28, 0xe3, 0x83, 0x97, 0xe3, 0x83, 0xad, 0xe3, 0x82, 0xb0, 0xe3, 0x83, 0xa9, 0xe3, 0x83, 0x9f, 0xe3, 0x83, 0xb3, 0xe3, 0x82, 0xb0, 0xe8, 0xa8, 0x80, 0xe8, 0xaa, 0x9e, 0x29, 0x22, 0x2c, 0x22, 0x77, 0x67, 0x43, 0x75, 0x72, 0x52, 0x65, 0x76, 0x69, 0x73, 0x69, 0x6f, 0x6e, 0x49, 0x64, 0x22, 0x3a, 0x37, 0x31, 0x36, 0x35, 0x33, 0x34, 0x37, 0x39, 0x2c, 0x22, 0x77, 0x67, 0x52, 0x65, 0x76, 0x69, 0x73, 0x69, 0x6f, 0x6e, 0x49, 0x64, 0x22, 0x3a, 0x37, 0x31, 0x36, 0x35, 0x33, 0x34, 0x37, 0x39, 0x2c, 0x22, 0x77, 0x67, 0x41, 0x72, 0x74, 0x69, 0x63, 0x6c, 0x65, 0x49, 0x64, 0x22, 0x3a, 0x32, 0x39, 0x31, 0x31, 0x35, 0x39, 0x31, 0x2c, 0x22, 0x77, 0x67, 0x49, 0x73, 0x41, 0x72, 0x74, 0x69, 0x63, 0x6c, 0x65, 0x22, 0x3a, 0x74, 0x72, 0x75, 0x65, 0x2c, 0x22, 0x77, 0x67, 0x49, 0x73, 0x52, 0x65, 0x64, 0x69, 0x72, 0x65, 0x63, 0x74, 0x22, 0x3a, 0x66, 0x61, 0x6c, 0x73, 0x65, 0x2c, 0x22, 0x77, 0x67, 0x41, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x22, 0x3a, 0x22, 0x76, 0x69, 0x65, 0x77, 0x22, 0x2c, 0x22, 0x77, 0x67, 0x55, 0x73, 0x65, 0x72, 0x4e, 0x61, 0x6d, 0x65, 0x22, 0x3a, 0x6e, 0x75, 0x6c, 0x6c, 0x2c, 0x22, 0x77, 0x67, 0x55, 0x73, 0x65, 0x72, 0x47, 0x72, 0x6f, 0x75, 0x70, 0x73, 0x22, 0x3a, 0x5b, 0x22, 0x2a, 0x22, 0x5d, 0x2c, 0x22, 0x77, 0x67, 0x43, 0x61, 0x74, 0x65, 0x67, 0x6f, 0x72, 0x69, 0x65, 0x73, 0x22, 0x3a, 0x5b, 0x22, 0xe3, 0x82, 0xa6, 0xe3, 0x82, 0xa3, 0xe3, 0x82, 0xad, 0xe3, 0x83, 0x9a, 0xe3, 0x83, 0x87, 0xe3, 0x82, 0xa3, 0xe3, 0x82, 0xa2, 0xe3, 0x81, 0xa8, 0xe3, 0x82, 0xa6, 0xe3, 0x82, 0xa3, 0xe3, 0x82, 0xad, 0xe3, 0x83, 0x87, 0xe3, 0x83, 0xbc, 0xe3, 0x82, 0xbf, 0xe3, 0x81, 0xa7, 0xe7, 0x95, 0xb0, 0xe3, 0x81, 0xaa, 0xe3, 0x82, 0x8b, 0xe5, 0x85, 0xac, 0xe5, 0xbc, 0x8f, 0xe3, 0x82, 0xa6, 0xe3, 0x82, 0xa7, 0xe3, 0x83, 0x96, 0xe3, 0x82, 0xb5, 0xe3, 0x82, 0xa4, 0xe3, 0x83, 0x88, 0x22, 0x2c, 0x22, 0x46, 0x4c, 0x4f, 0x53, 0x53, 0xe9, 0x96, 0xa2, 0xe9, 0x80, 0xa3, 0xe3, 0x81, 0xae, 0xe3, 0x82, 0xb9, 0xe3, 0x82, 0xbf, 0xe3, 0x83, 0x96, 0xe9, 0xa0, 0x85, 0xe7, 0x9b, 0xae, 0x22, 0x2c, 0x22, 0xe3, 0x82, 0xaa, 0xe3, 0x83, 0x96, 0xe3, 0x82, 0xb8, 0xe3, 0x82, 0xa7, 0xe3, 0x82, 0xaf, 0xe3, 0x83, 0x88, 0xe6, 0x8c, 0x87, 0xe5, 0x90, 0x91, 0xe8, 0xa8, 0x80, 0xe8, 0xaa, 0x9e, 0x22, 0x2c, 0x22, 0xe6, 0x95, 0xb0, 0xe5, 0x80, 0xa4, 0xe7, 0xb7, 0x9a, 0xe5, 0xbd, 0xa2, 0xe4, 0xbb, 0xa3, 0xe6, 0x95, 0xb0, 0x22, 0x2c, 0x22, 0xe4, 0xb8, 0xa6, 0xe5, 0x88, 0x97, 0xe3, 0x82, 0xb3, 0xe3, 0x83, 0xb3, 0xe3, 0x83, 0x94, 0xe3, 0x83, 0xa5, 0xe3, 0x83, 0xbc, 0xe3, 0x83, 0x86, 0xe3, 0x82, 0xa3, 0xe3, 0x83, 0xb3, 0xe3, 0x82, 0xb0, 0x22, 0x2c, 0x22, 0xe3, 0x83, 0x97, 0xe3, 0x83, 0xad, 0xe3, 0x82, 0xb0, 0xe3, 0x83, 0xa9, 0xe3, 0x83, 0x9f, 0xe3, 0x83, 0xb3, 0xe3, 0x82, 0xb0, 0xe8, 0xa8, 0x80, 0xe8, 0xaa, 0x9e, 0x22, 0x2c, 0x22, 0x4c, 0x69, 0x6e, 0x75, 0x78, 0xe5, 0x90, 0x91, 0xe3, 0x81, 0x91, 0xe6, 0x95, 0xb0, 0xe5, 0x80, 0xa4, 0xe8, 0xa7, 0xa3, 0xe6, 0x9e, 0x90, 0xe3, 0x82, 0xbd, 0xe3, 0x83, 0x95, 0xe3, 0x83, 0x88, 0xe3, 0x82, 0xa6, 0xe3, 0x82, 0xa7, 0xe3, 0x82, 0xa2, 0x22, 0x5d, 0x2c, 0x22, 0x77, 0x67, 0x42, 0x72, 0x65, 0x61, 0x6b, 0x46, 0x72, 0x61, 0x6d, 0x65, 0x73, 0x22, 0x3a, 0x66, 0x61, 0x6c, 0x73, 0x65, 0x2c, 0x22, 0x77, 0x67, 0x50, 0x61, 0x67, 0x65, 0x43, 0x6f, 0x6e, 0x74, 0x65, 0x6e, 0x74, 0x4c, 0x61, 0x6e, 0x67, 0x75, 0x61, 0x67, 0x65, 0x22, 0x3a, 0x22, 0x6a, 0x61, 0x22, 0x2c, 0x22, 0x77, 0x67, 0x50, 0x61, 0x67, 0x65, 0x43, 0x6f, 0x6e, 0x74, 0x65, 0x6e, 0x74, 0x4d, 0x6f, 0x64, 0x65, 0x6c, 0x22, 0x3a, 0x22, 0x77, 0x69, 0x6b, 0x69, 0x74, 0x65, 0x78, 0x74, 0x22, 0x2c, 0x22, 0x77, 0x67, 0x53, 0x65, 0x70, 0x61, 0x72, 0x61, 0x74, 0x6f, 0x72, 0x54, 0x72, 0x61, 0x6e, 0x73, 0x66, 0x6f, 0x72, 0x6d, 0x54, 0x61, 0x62, 0x6c, 0x65, 0x22, 0x3a, 0x5b, 0x22, 0x22, 0x2c, 0x22, 0x22, 0x5d, 0x2c, 0x22, 0x77, 0x67, 0x44, 0x69, 0x67, 0x69, 0x74, 0x54, 0x72, 0x61, 0x6e, 0x73, 0x66, 0x6f, 0x72, 0x6d, 0x54, 0x61, 0x62, 0x6c, 0x65, 0x22, 0x3a, 0x5b, 0x22, 0x22, 0x2c, 0x22, 0x22, 0x5d, 0x2c, 0x22, 0x77, 0x67, 0x44, 0x65, 0x66, 0x61, 0x75, 0x6c, 0x74, 0x44, 0x61, 0x74, 0x65, 0x46, 0x6f, 0x72, 0x6d, 0x61, 0x74, 0x22, 0x3a, 0x22, 0x6a, 0x61, 0x22, 0x2c, 0x22, 0x77, 0x67, 0x4d, 0x6f, 0x6e, 0x74, 0x68, 0x4e, 0x61, 0x6d, 0x65, 0x73, 0x22, 0x3a, 0x5b, 0x22, 0x22, 0x2c, 0x22, 0x31, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x32, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x33, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x34, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x35, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x36, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x37, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x38, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x39, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x31, 0x30, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x31, 0x31, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x31, 0x32, 0xe6, 0x9c, 0x88, 0x22, 0x5d, 0x2c, 0x22, 0x77, 0x67, 0x4d, 0x6f, 0x6e, 0x74, 0x68, 0x4e, 0x61, 0x6d, 0x65, 0x73, 0x53, 0x68, 0x6f, 0x72, 0x74, 0x22, 0x3a, 0x5b, 0x22, 0x22, 0x2c, 0x22, 0x31, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x32, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x33, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x34, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x35, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x36, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x37, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x38, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x39, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x31, 0x30, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x31, 0x31, 0xe6, 0x9c, 0x88, 0x22, 0x2c, 0x22, 0x31, 0x32, 0xe6, 0x9c, 0x88, 0x22, 0x5d, 0x2c, 0x22, 0x77, 0x67, 0x52, 0x65, 0x6c, 0x65, 0x76, 0x61, 0x6e, 0x74, 0x50, 0x61, 0x67, 0x65, 0x4e, 0x61, 0x6d, 0x65, 0x22, 0x3a, 0x22, 0x4a, 0x75, 0x6c, 0x69, 0x61, 0x5f, 0x28, 0xe3, 0x83, 0x97, 0xe3, 0x83, 0xad, 0xe3, 0x82, 0xb0, 0xe3, 0x83, 0xa9, 0xe3, 0x83, 0x9f, 0xe3, 0x83, 0xb3, 0xe3, 0x82, 0xb0, 0xe8, 0xa8, 0x80, 0xe8, 0xaa, 0x9e, 0x29, 0x22, 0x2c, 0x22, 0x77, 0x67, 0x52, 0x65, 0x6c, 0x65, 0x76, 0x61, 0x6e, 0x74, 0x41, 0x72, 0x74, 0x69, 0x63, 0x6c, 0x65, 0x49, 0x64, 0x22, 0x3a, 0x32, 0x39, 0x31, 0x31, 0x35, 0x39, 0x31, 0x2c, 0x22, 0x77, 0x67, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x49, 0x64, 0x22, 0x3a, 0x22, 0x58, 0x48, 0x4c, 0x35, 0x6d, 0x67, 0x70, 0x41, 0x41, 0x44, 0x30, 0x41, 0x41, 0x4a, 0x79, 0x7a, 0x4f, 0x6f, 0x45, 0x41, 0x41, 0x41, 0x43, 0x4b, 0x22, 0x2c, 0x22, 0x77, 0x67, 0x43, 0x53, 0x50, 0x4e, 0x6f, 0x6e, 0x63, 0x65, 0x22, 0x3a, 0x66, 0x61, 0x6c, 0x73, 0x65, 0x2c, 0x22, 0x77, 0x67, 0x49, 0x73, 0x50, 0x72, 0x6f, 0x62, 0x61, 0x62, 0x6c, 0x79, 0x45, 0x64, 0x69, 0x74, 0x61, 0x62, 0x6c, 0x65, 0x22, 0x3a, 0x74, 0x72, 0x75, 0x65, 0x2c, 0x22, 0x77, 0x67, 0x52, 0x65, 0x6c, 0x65, 0x76, 0x61, 0x6e, 0x74, 0x50, 0x61, 0x67, 0x65, 0x49, 0x73, 0x50, 0x72, 0x6f, 0x62, 0x61, 0x62, 0x6c, 0x79, 0x45, 0x64, 0x69, 0x74, 0x61, 0x62, 0x6c, 0x65, 0x22, 0x3a, 0x74, 0x72, 0x75, 0x65, 0x2c, 0x22, 0x77, 0x67, 0x52, 0x65, 0x73, 0x74, 0x72, 0x69, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x45, 0x64, 0x69, 0x74, 0x22, 0x3a, 0x5b, 0x5d, 0x2c, 0x22, 0x77, 0x67, 0x52, 0x65, 0x73, 0x74, 0x72, 0x69, 0x63, 0x74, 0x69, 0x6f, 0x6e, 0x4d, 0x6f, 0x76, 0x65, 0x22, 0x3a, 0x5b, 0x5d, 0x2c, 0x22, 0x77, 0x67, 0x43, 0x61, 0x74, 0x65, 0x67, 0x6f, 0x72, 0x79, 0x54, 0x72, 0x65, 0x65, 0x50, 0x61, 0x67, 0x65, 0x43, 0x61, 0x74, 0x65, 0x67, 0x6f, 0x72, 0x79, 0x4f, 0x70, 0x74, 0x69, 0x6f, 0x6e, 0x73, 0x22, 0x3a, 0x22, 0x7b, 0x5c, 0x22, 0x6d, 0x6f, 0x64, 0x65, 0x5c, 0x22, 0x3a, 0x30, 0x2c, 0x5c, 0x22, 0x68, 0x69, 0x64, 0x65, 0x70, 0x72, 0x65, 0x66, 0x69, 0x78, 0x5c, 0x22, 0x3a, 0x32, 0x30, 0x2c, 0x5c, 0x22, 0x73, 0x68, 0x6f, 0x77, 0x63, 0x6f, 0x75, 0x6e, 0x74, 0x5c, 0x22, 0x3a, 0x74, 0x72, 0x75, 0x65, 0x2c, 0x5c, 0x22, 0x6e, 0x61, 0x6d, 0x65, 0x73, 0x70, 0x61, 0x63, 0x65, 0x73, 0x5c, 0x22, 0x3a, 0x66, 0x61, 0x6c, 0x73, 0x65,
...(略)
]

となり,取得は出来ている模様

感想

課題が２つ浮かんだので、良かった

1.fieldsnameでとったfieldに:がつく問題(これがなければforで回せた)

eg):status,:body これ:を取りたい

2.bodyで取得したUInt8配列を文字列に直したい

楽しかった

参考にしたURL,書籍

Julia Programming Projects: Learn Julia 1.x by building apps for data analysis, visualization, machine learning, and the web (English Edition)

JuliaでJSON操作

動機

いずれJuliaでWebアプリを作るから
PythonでJSONを扱うことが多いので、Juliaでもやりたくなったから

目標

JuliaでJSON操作

作業環境

今回の環境(前回と同様)

julia> versioninfo()
Julia Version 1.1.0
...

JSONをパース

(v1.1) pkg>add JSON
julia> edit("json_parse.jl")

json_parse.jlが作成され、エディタが自動で開くので、次のように編集

using JSON

first_json = """{
        "key":"value",
        "number":8,
        "array":[2,3,5,7,11],
        "dict":{"k1":"val1","k2":2}
}"""

print(first_json)
print("JSONパースしてみる")
print(JSON.parse(first_json))

シェルで実行

shell> julia json_parse.jl
{
        "key":"value",
        "number":8,
        "array":[2,3,5,7,11],
        "dict":{"k1":"val1","k2":2}
}JSONパースしてみるDict{String,Any}("key"=>"value","dict"=>Dict{String,Any}("k1"=>"val1","k2"=>2),"number"=>8,"array"=>Any[2, 3, 5, 7, 11])

Dict{String,Any}→JSONなのでkeyはString

辞書型をJSONに変換,JSONファイルの作成

lets_make_json.jlを次の様に決めて実行すると

using JSON

see_json_key = Dict{Int64,Union{Int64,String}}(1=>"text", 2=>"hello",3=>1111)

print(JSON.json(see_json_key))

f = open("hello.json", "w")
#hello.jsonを作成

JSON.print(f, see_json_key)
#hello.jsonにsee_json_keyを書き込む

close(f)
#閉じる

1.keyがStringに勝手になっていること
2.hello.jsonが作成され,書き込みが出来ていること
の２つを確認する

shell> julia lets_make_json.jl
{"2":"hello","3":1111,"1":"text"}
shell> cat hello.json
{"2":"hello","3":1111,"1":"text"}

できた！

感想

ほぼPythonだった

参考URL,書籍

Julia 1.0 Programming Cookbook: Over 100 numerical and distributed computing recipes for your daily data science workflow (English Edition)

最近日本語訳が出たみたいです

JuliaでWebページ作ってみる(Genie.jl)1/n

動機

Juliaが好きだから

目標

Genie.jlのハローワールドをやる,つまり
/helloに「Hello World!」の表示

作業環境

今回の環境(atomにJuno入れてます,Ubuntu18.04)

julia> versioninfo()
Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_EDITOR = atom -a
  JULIA_NUM_THREADS = 4
  JULIA_REVISE = auto

Genie.jlについて

公式によると(http://genieframework.com/)
The highly productive Julia web framework Genie is a full-stack MVC web framework that promotes a streamlined and efficient workflow for developing modern web applications.
らしいです
公式レポジトリのログを見ると,頻繁に更新されているようです
Genieの作者のAdrian Salceanu様はバルセロナのJuliaユーザーグーループのオーガナイザのようで,最近 Julia Programming Projects: Learn Julia 1.x by building apps for data analysis, visualization, machine learning, and the web (English Edition)などの本を出版されています
(まだ読んでる途中なので,後日レビューします)
ご本人のJuliacon登壇動画によると
"Like Django and Rails, but without the fails..."
だそうです
良さそうですね

MVCモデルとは

Model:ビジネスロジックを担当
View:表示,入出力担当
Controller:ユーザーの入力に基づき，ModelとViewを制御
が揃っていて、それぞれ役割を分割してコーディングするモデルを指すようです

Genieのインストールとサーバーの起動

julia>

]を押す,するとJuliaのヴァージョン管理ツールpkgのモードに入る,]は表示されない

pkg>

0212という名前の仮想環境を作って、入る
(conda create -n 0212して、sour activate 0212するような)

pkg>activate 0212

Genieのインストール

(0212) pkg>add https://github.com/essenciary/Genie.jl

Backspaceでpkgモードを抜けて,Genieを使う宣言をする

julia>using Genie

;(セミコロン)でshellモードに入って,0212に入る

shell>cd 0212

Backspaceでshellモードを抜けて test_appという名前で作成して、起動

julia>Genie.REPL.new_app("test_app")

julia>Genie.AppServer.startup()

http://localhost:8000 にアクセス

f:id:julialangisthebestlang:20190212042051p:plain
welcome.html
とりあえず無事起動したよう

Genie Hello World

test_appの構造を見てみる

shell> tree test_app
test_app
├── LICENSE.md
├── Manifest.toml
├── Project.toml
├── README.md
├── app
│   ├── assets
│   │   ├── css
│   │   │   └── application.css
│   │   ├── fonts
│   │   └── js
│   │       ├── application.js
│   │       └── channels.js
│   ├── helpers
│   │   ├── ValidationHelper.jl
│   │   └── ViewHelper.jl
│   ├── layouts
│   │   └── app.jl.html
│   └── resources
├── bin
│   ├── repl
│   └── server
├── build
├── cache
├── config
│   ├── database.yml
│   ├── env
│   │   ├── dev.jl
│   │   ├── prod.jl
│   │   └── test.jl
│   ├── global.jl
│   ├── initializers
│   │   ├── converters.jl
│   │   └── searchlight.jl
│   ├── routes.jl
│   └── secrets.jl
├── db
│   ├── migrations
│   └── seeds
├── docs
│   ├── make.jl
│   ├── mkdocs.yml
│   └── src
│       └── index.md
├── env.jl
├── genie.jl
├── lib
├── log
│   └── dev.log
├── package.json
├── public
│   ├── css
│   │   ├── bootstrap-theme.css
│   │   ├── bootstrap-theme.min.css
│   │   ├── bootstrap.css
│   │   ├── bootstrap.min.css
│   │   ├── dosis-font.css
│   │   ├── prism.css
│   │   ├── style.css
│   │   └── themify-icons.css
│   ├── error-404.html
│   ├── error-500.html
│   ├── favicon.ico
│   ├── favicon.png
│   ├── fonts
│   │   ├── dosis
│   │   │   ├── 3isE9muMMOq1K7TQ7HkKvIDGDUGfDkXyfkzVDelzfFk.woff2
│   │   │   ├── O6SOu9hYsPHTU43R17NS5XYhjbSpvc47ee6xR_80Hnw.woff2
│   │   │   ├── RPKDmaFi75RJkvjWaDDb0nYhjbSpvc47ee6xR_80Hnw.woff2
│   │   │   ├── VK-RlLrn4NFhRGqPkj6IwBkAz4rYn47Zy2rvigWQf6w.woff2
│   │   │   └── oaBFj7Fz9Y9_eW3k9Jd9X6CWcynf_cDxXwCLxiixG1c.woff2
│   │   ├── glyphicons-halflings-regular.eot
│   │   ├── glyphicons-halflings-regular.svg
│   │   ├── glyphicons-halflings-regular.ttf
│   │   ├── glyphicons-halflings-regular.woff
│   │   ├── glyphicons-halflings-regular.woff2
│   │   ├── themify.eot
│   │   ├── themify.svg
│   │   ├── themify.ttf
│   │   └── themify.woff
│   ├── img
│   │   ├── community.png
│   │   ├── contribute-2.png
│   │   ├── docs.png
│   │   ├── genie-sad.png
│   │   └── genie.png
│   ├── js
│   │   ├── ansi_up.js
│   │   ├── ansi_up.js.map
│   │   ├── bootstrap.js
│   │   ├── bootstrap.min.js
│   │   ├── jquery.easing.min.js
│   │   ├── jquery.min.js
│   │   ├── jquery.min.map
│   │   ├── prism.js
│   │   ├── scrolling-nav.js
│   │   ├── static.js
│   │   └── validator.js
│   ├── robots.txt
│   └── welcome.html
├── session
├── src
│   ├── App.jl
│   └── Toolbox.jl
├── task
├── test
│   └── test_config.jl
└── webpack.config.js

31 directories, 76 files

JULIA_EDITORで指定したエディタでconfig内のroutes.jlを編集する

julia>edit("config/routes.jl")

(私の場合)atomが開いて、route.jlが開く

using Genie.Router

route("/") do
  serve_static_file("/welcome.html")
end

ページを足して、Hello World!を表示させるために,routes.jlに次を追記

route("/hello") do
  "Hello World!"
end

http://localhost:8000/hello にアクセス

f:id:julialangisthebestlang:20190212042105p:plain
こんな感じ
できた！

感想等

次回は認証とかやろうと思います

参考にしたURL,書籍

MVCモデルについて--qiita
Genie.jl--Github
Julia Programming Projects: Learn Julia 1.x by building apps for data analysis, visualization, machine learning, and the web (English Edition)

日本語版