Posts match “ nodejs ” tag:

授權方式(Auhorization): CC-BY 4.0

Intro

利用meteor.js找回以前的手感,接下來要找個可以做成專案的東西。

架構

  • Backend: Express based on node.js
  • Frontend: Angular.js
  • wireframe: moqups
  • Layout: Bootstrap
  • DB: MongoDB
  • VC: bitbucket
  • 其他: Grunt, Berkshelf, Vagrant, travisCI, jade, ..blah

Express

首先參照這篇express tutorial Setup環境,

安裝Express

$ sudo npm install express -g
express@3.3.1 /usr/local/lib/node_modules/express
├── methods@0.0.1
├── fresh@0.1.0
├── range-parser@0.0.4
├── cookie-signature@1.0.1
├── buffer-crc32@0.2.1
├── cookie@0.1.0
├── debug@0.7.2
├── commander@0.6.1
├── mkdirp@0.3.4
├── send@0.1.1 (mime@1.2.9)
└── connect@2.8.1 (uid2@0.0.2, pause@0.0.1, qs@0.6.5, bytes@0.2.0, cookie@0.0.5, formidable@1.0.14)

安裝好了 express後

建立專案

$ express order-plant --sessions --css less
   create : order-plant
   create : order-plant/package.json
   create : order-plant/app.js
   create : order-plant/public
   create : order-plant/public/images
   create : order-plant/public/javascripts
   create : order-plant/public/stylesheets
   create : order-plant/public/stylesheets/style.less
   create : order-plant/routes
   create : order-plant/routes/index.js
   create : order-plant/routes/user.js
   create : order-plant/views
   create : order-plant/views/layout.jade
   create : order-plant/views/index.jade

   install dependencies:
     $ cd order-plant && npm install

   run the app:
     $ node app

因為打算用Bootsrap, 所以這邊css engine我選擇less

Post Install

$ cd order-plant && npm install
express@3.3.1 node_modules/express
├── methods@0.0.1
├── fresh@0.1.0
├── range-parser@0.0.4
├── cookie-signature@1.0.1
├── buffer-crc32@0.2.1
├── cookie@0.1.0
├── debug@0.7.2
├── commander@0.6.1
├── mkdirp@0.3.4
├── send@0.1.1 (mime@1.2.9)
└── connect@2.8.1 (uid2@0.0.2, pause@0.0.1, qs@0.6.5, bytes@0.2.0, cookie@0.0.5, formidable@1.0.14)

jade@0.32.0 node_modules/jade
├── character-parser@1.0.2
├── mkdirp@0.3.5
├── commander@1.2.0 (keypress@0.1.0)
├── monocle@0.1.48 (readdirp@0.2.5)
├── with@1.1.0 (uglify-js@2.3.6)
├── constantinople@1.0.1 (uglify-js@2.3.6)
└── transformers@2.0.1 (css@1.0.8, promise@2.0.0, uglify-js@2.2.5)

less-middleware@0.1.12 node_modules/less-middleware
├── mkdirp@0.3.5
└── less@1.4.0 (mime@1.2.9, ycssmin@1.0.1, request@2.21.0)

Run App

$ node app.js

此時就可以看到Express的歡迎畫面

修改package.json

  1 {
  2   "name": "order-plant",
  3   "version": "0.0.1",
  4   "private": true,
  5   "scripts": {
  6     "start": "node app.js"
  7   },
  8   "dependencies": {
  9     "express": "3.3.1",
 10     "jade": "*",
 11     "less-middleware": "*"
 12   }
 13 }
 ```
 
 ## Commit to Bitbucket
1. 在bitbucket建立好帳號
2. create private repo
3. 產生ssh key
4. 需要ssh-agent
5. ssh-add
6. ssh-add -l && ssh -T git@bitbucket.org  確認

 ```
 $ git init
 $ git remote add origin git://repo
 $ git add *
 $ git push origin --all

授權方式(Auhorization): CC-BY 4.0

Introduction

突然發覺自己事情其實很要求,雖然沒有說完美,但至少要按照一些標準,總之就是要打造自己的rest api停止去直接連資料庫。
這邊我們使用nodejs + expressjs + mysql來完成要求

Data Format

*id, room, time, value1, value2, ..., value8

First URL Routing

既然是rest api就要考慮好url routing
從Data Format可以規劃出以下的url route
http://api.example.com/room/start_datetime/end_datetime/value*
實際上調用情況也要先想好
e.g.:

  1. http://api.example.com/room1/2013-05-07%2011:00/2013-05-08%2012:00/value1
  2. http://api.example.com/room2/2013-05-07%2011:00/2013-05-08%2012:00/all

JSON format Callback

事先規劃好回傳的JSON Foramt
{
time: 2013-05-07 11:00:00
value1: 231
}

Implement

api.js

var mysql = require('mysql')

var client = mysql.createConnection({...});

exports.findRecord = function (req, res) {
    var room = req.params.room,
        startDate = req.params.startDate,
        endDate = req.params.endDate,
        data = req.params.data;
  // 處理 params
    sql = "select ....";

  client.query(sql, function (err, rows, fields) {
    res.jsonp(rows);
  });
};

app.js

把api.js包進來

api = require('./routes/api');

設定route path

app.get('/:room/:startDate/:endDate/:data', api.findRecord);

基本上這樣就完成了 REST API!
非常的簡單 :)

ps 記得要在package.json裡面設好"dependencies

授權方式(Auhorization): CC-BY 4.0

Intro

最近在對付某個Big5網站, 用js來寫抓抓, 對於這種古老網站, 一定都會碰到big5 to utf-8問題

nodejs上面有兩套node-iconv跟iconv-lite, 前者是原生支援iconv, 後者是用js硬幹出來的

一開始用了前面那個卻轉不出效果, 用後面的終於成功了,

var http = require('http')
  , program = require('commander')
  , iconv = require('iconv-lite')
  , BufferHelper = require('bufferhelper')
  , options = {
    host: 'www.cpbl.com.tw',
    port: '80',
    path: '/' };
    
 http.get(options, function (res) {
      var bufferhelper = new BufferHelper();
      res.on('data', function (chunk) {
        bufferhelper.concat(chunk);
      });
      res.on('end', function (){
        console.log(iconv.decode(bufferhelper.toBuffer(), 'Big5'));
      });
    });

用request套件, 似乎會強制幫你轉utf-8, 所以要用http.get然後處理回來的buffer

授權方式(Auhorization): CC-BY 4.0

English Version

準備

要先申請google的api key
統一到 google developer console

  1. create project 填name和ID
  2. API auth 找到youtube data api 把status 設成 ON
  3. Credentials 申請一組Client ID for web application 擁有Client ID和Client secret準備工作就完成了 !! 注意youtube live streaming api不支援service account,原因是因為必須要存取使用者權限,所以除了service account flow以外都支援。

APIs Explorer

google提供的api 都可以先用api explorer來玩
https://developers.google.com/apis-explorer/

  1. youtube Data API 找到youtube.liveBroadcasts.insert
  2. 將OAuth request 設成 ON
  3. part欄位 snippet,status
  4. request body欄位 可以用add property選擇status跟snippet
    request body
    {
    "snippet": {
    "scheduledEndTime": "2014-04-16T20:00:00.0+08",
    "scheduledStartTime": "2014-04-16T18:00:00.0+08",
    "title": "hero g0v"
    },
    "status": {
    "privacyStatus": "private"
    }
    }
    
    以上都是必填欄位, 注意時間格式為ISO-8601: YYYY-MM-DDThh:mm:ss.sZ

nodejs

google有提供nodejs 的api client
source code
一般熟悉nodejs的人直接npm i googleapis就可以了

流程

  1. 使用Oauth 取得 authorization code
  2. 用auhorization code取得access token
  3. 利用access token存取youtube api
  4. 以下是youtube live stream api的五個步驟
    1. Set up your broadcast
      1. create broadcast (新增現場直播)
      2. create stream (創建串流)
      3. bind broadcast and stream (選取串流)
    2. Claim your content # 這步驟需要取得Content ID API, 沒有的話就是沒辦法設定一些著作權的policy而已
    3. Test (預覽)
      • liveStreams.list 去找剛剛那個stream id, status變成 active時
      • 呼叫livebroadcasts.transition 'testing'
    4. Broadcast (啟動串流直播)
      • 當 test 完成呼叫 呼叫livebroadcasts.transition 'live'
    5. Conclude your broadcast 完成!

步驟詳細內容請參考 Life of a broadcast

新增現場直播

必要參數

part: [snippet, status]
snippet: [title,scheduledStartTime,scheduledEndTime]
status: [privacyStatus]

err, broadcast <- client.youtube.live-broadcasts.insert part: 'snippet,status', req-broadcast
  .with-auth-client auth
  .execute
return console.log err if err
console.log broadcast

目前會看到成功回傳

response body
{ kind: 'youtube#liveBroadcast',
  etag: '"X98aQHqGvPBJLZLOiSGUHCM9jnE/2kEP-c4VUkPHY301QBKEBcJJRMA"',
  id: 'v-MKk0Z3HB4',
  snippet:
   { publishedAt: '2014-04-16T05:23:58.000Z',
     channelId: 'UCWDNcRqKYS8PjGn2Z7RUocg',
     title: 'test',
     description: '',
     thumbnails: { default: [Object], medium: [Object], high: [Object] },
     scheduledStartTime: '2014-05-16T21:00:00.000Z',
     scheduledEndTime: '2014-05-17T00:00:00.000Z' },
  status:
   { lifeCycleStatus: 'created',
     privacyStatus: 'private',
     recordingStatus: 'notRecording' } }

可以到 youtube 活動現場看到剛剛新增的event

創建串流

必要參數

part: [snippet, cdn]
snippet: [title]
cdn: [format, ingestionType]

err, stream <- client.youtube.live-streams.insert part: 'snippet,cdn', req-stream
  .with-auth-client auth
  .execute
return console.log err if err
console.log stream

目前會看到成功回傳

livesteam response body
{

 "kind": "youtube#liveStream",
 "etag": "\"X98aQHqGvPBJLZLOiSGUHCM9jnE/F8dDAvNbcjro8PFQe4H82hilSmQ\"",
 "id": "WDNcRqKYS8PjGn2Z7RUocg1397629864085436",
 "snippet": {
  "publishedAt": "2014-04-16T06:31:04.000Z",
  "channelId": "UCWDNcRqKYS8PjGn2Z7RUocg",
  "title": "test g0v",
  "description": ""
 },
 "cdn": {
  "format": "720p",
  "ingestionType": "rtmp",
  "ingestionInfo": {
   "streamName": "ly.forever.rkx0-bwp3-7fy0-9ykd",
   "ingestionAddress": "rtmp://a.rtmp.youtube.com/live2",
   "backupIngestionAddress": "rtmp://b.rtmp.youtube.com/live2?backup=1"
  }
 }
}

選取串流

必要參數

part: [id,contentDetails]
id: [broadcast id]
streamId: [stream id]

err, bind <- client.youtube.live-broadcasts.bind part: 'id,contentDetails', id: video-id, stream-id: stream-id
  .with-auth-client auth
  .execute
console.log bind

目前成功會回傳

livebroadcast.bind response body
{

 "kind": "youtube#liveBroadcast",
 "etag": "\"X98aQHqGvPBJLZLOiSGUHCM9jnE/B5DR-1fKyXag5mlIwZ6Tqc9-ysk\"",
 "id": "v-MKk0Z3HB4"
}

使用ffmpeg 將影片傳到串流上

new fluent-ffmpeg source: 'rtmp://cp49989.live.edgefcs.net:1935/live/streamRM1@2564'
  .with-video-codec 'libx264'
  .with-audio-codec 'libfaac'
  .with-audio-bitrate '128k'
  .with-audio-channels 1
  .with-audio-frequency 44100
  .with-size '426x240'
  .with-fps 30
  .to-format 'flv'
  .add-options ['-g 1', '-force_key_frames 2']
  .on 'start' -> console.log 'FFmpeg start with ' + it
  .on 'progress' ->
    err, streams <- client.youtube.live-streams.list part: 'id,status', id: stream-id
      .with-auth-client auth
      .execute
    err, test <- transit-it auth, client, 'testing'
    err, live <- transit-it auth, client, 'live'
    console.log live.status if live
  .on 'end' -> console.log 'FFmpeg end.'
  .write-to-stream stream-url

測試直播

err, it <- client.youtube.live-broadcasts.transition broadcast-status: 'testing', id: video-id, part: 'id,status,contentDetails'
    .with-auth-client auth
    .execute
consoel.log it

啟動串流直播

err, it <- client.youtube.live-broadcasts.transition broadcast-status: 'live', id: video-id, part: 'id,status,contentDetails'
    .with-auth-client auth
    .execute
consoel.log it

完整程式碼

ffmpeg的詳細指令

$ ffmpeg -i rtmp://cp49989.live.edgefcs.net:1935/live/streamRM1@2564 -vcodec libx264 -maxrate 700k -r 30 -s 426x240 -g 0 -acodec libvo_aacenc -ab 128k -ac 1 -ar 44100 -f flv rtmp://

$ ffmpeg -i rtmp://h264media02.ly.gov.tw:1935/live07/300K_Live -vcodec libx264 -maxrate 700k -minrate 300k -r 30 -s 426x240 -g 1 -force_key_frames 2 -acodec libvo_aacenc -ab 128k -ac 1 -ar 44100 -f flv rtmp://

心得

google的document很完整,但還是容易霧裡看花,一下就迷失了。
至少比irccloud還要去猜api好多了~不愧是google~
但是nodejs sample在document裡面都沒有,幸好github上面的都很完整!
googleapis example
經過這次經驗,最後有抓到如何快速看google api document的訣竅了 :)