[Golang][程式設計週記].. 2015第九週

##雜七雜八感言:

最近搞完一些Golang部分,有開始跑各個平台的cross platform的building solution.果然C++跨平台一點都不簡單,真是80%設定環境,20%寫跨平台的code….

##筆記:

[Golang] 關於Golang 有趣的網頁與小工具

[Apple] 關於 Apple 03/09 發表會

這個禮拜讓蘋果迷最興奮的就是三月的發表會. 裡面有包括了ResearchKitMacBook更新跟Apple Watch. 兩個跟硬體有關的就沒啥好評論….

ResearchKit 與 HealthKit

根據已經放出的消息,ResearchKit透過兩項(目前已知是手指點擊與聲音)的手機功能,可以提供一些簡單的診斷.並且把這些資料提供給醫療研究機關. 等到詳細資料出來應該會更仔細地研究一下,是否一般廠商有介入的空間.

[Android] 在Android Stuido 1.1 上面使用

自從Google 推出 Android Studio 正式版之後,自然而然也開始把Eclipse的力道放輕. 於是要弄新的SDK也變成是一個工程.

所以現在要開發新的cross platform module,所以也必須要在 Android Studio + NDK + JNI 去執行跨平台的 C++ module.

根據最新版的Android Studio 1.1 如果依照著一般的方式來部署NDK會發現以下的問題:

    WARNING [Project: :app] Current NDK support is deprecated.  Alternative will be provided in the future. 

找了一堆論壇,其實並沒有一個有系統地整理,直到看到這一篇文章,以下把方法整理一下:

  • 安裝 Android Studio 1.1
  • 下載並且安裝 Android NDK r10d
  • 先到一開始的畫面[Configure]->[settings]->[External Tools] 設定Gradle
  • 設定javah
  • 新增一個NDK Build 類別為NDK
    • Program:C:\ndk\ndk-build.cmd
    • Parameters:NDK_PROJECT_PATH=$ModuleFileDir$/build/intermediates/ndk NDK_LIBS_OUT=$ModuleFileDir$/src/main/jniLibs NDK_APPLICATION_MK=$ModuleFileDir$/src/main/jni/Application.mk APP_BUILD_SCRIPT=$ModuleFileDir$/src/main/jni/Android.mk V=1
    • Working directory:$SourcepathEntry$
  • 這樣設定好就可以
  • 複製ndk-r10d 目錄下的sample\hello-jni來使用
  • 會出現一個[c]在上面按下 右鍵跑 “NDK Build”,來製造出.so
  • [App]->[build.gradle]新增:

      sourceSets.main.jni.srcDirs = []
    
  • 然後sync gradle 這時候會發現[C]不見了,就可以跑App

之後要切換就是把上面的那段 comment掉才能build NDK

[Android/Golang]Server-Side In-App-Purchase Verification with Google Play on GO

image

Preface

Working on Android application development, you will need to have IAP (In-App Purchase) items. Normally it is simple, if your application is standalone not connect to any server.

If you app need connect to server for IAP items (such as game server, database service …), it might have risk that here might be a fake app (or crack app) to fake the purchase command in your app to get privilege action or items.

In this case, our server will need to do a sever-to-server side certification with Google Play.

Android has done great documentation in their Android portal. But it separate into different part, so I am trying to summarized it here.

Hope it help.

Google Service Entrypoint - Google API

When we want to communication with any Google Service, the only entry point is using Google API Console..

So, here let’s start to connect to Google API.

image

According to Google spec, this diagram describe about how to communication with Google API via HTTP/REST.

However it might be easy to communication via OAuth2 in Golang.

Google API certification flow:

Before jump into Google API function call. Please take a look about OAuth2 server account document. Here is the basic diagram come from Google Document, it is the basic concept if you want to using OAuth2 with Google API from Web Application to using client (end-user) Google Account.

However, in our case. Our server could using Google API directly if we already have a account in Google developer console.

Here is the simple flow:

  • Create server related google API account in Google developer console.
  • Request P12 key or Json file to setup client.
  • Server client using Google OAuth2 to connect Google API and query related result.

Let’s start a simple case to make it more clear.

Sample to connect to Google API with Golang OAuth2

In this case, we will enable one simple API service call “Google Contacts API”

image

  • Enable Google Play Console API to Authentication. Refer here
  • Using Service OAuth2 solution here
  • It have two ways to retrieval data one is using p12 key, another is config json file from Google Console Panel.
  • To get json file from Google Console:
    • Enter Google Console
    • Choose “Project” (or create new one)
    • Make sure you enable the API you need. (ex:Contacts API)
    • Choose “credential” -> “Create New Client ID”
    • Choose “Server Account”
    • Click “Generate New JSON key” to download json file or click “Generate new P12 Key” to download key.
  • Here is a sample to use json file to setup oauth2.

      package main
        
      import (
      	"golang.org/x/oauth2"
      	"golang.org/x/oauth2/google"
      	"io/ioutil"
      	"log"
      )
        
      func main() {
      	//Note: your json file need to put together with this program.
      	data, err := ioutil.ReadFile("downloaded.json")
      	if err != nil {
      		log.Fatal(err)
      	}
      	conf, err := google.JWTConfigFromJSON(data, "https://www.google.com/m8/feeds")
      	if err != nil {
      		log.Fatal(err)
      	}
      	client := conf.Client(oauth2.NoContext)
      	res, err := client.Get("https://www.google.com/m8/feeds/contacts/default/full")
      	if err != nil {
      		log.Println("err:", err)
      	}
      	body, err := ioutil.ReadAll(res.Body)
      	log.Println(string(body))
      }
    

Here is a response sample from Google Contact API

    <feed xmlns='http://www.w3.org/2005/Atom'
        xmlns:openSearch='http://a9.com/-/spec/opensearch/1.1/'
        xmlns:gContact='http://schemas.google.com/contact/2008'
        xmlns:batch='http://schemas.google.com/gdata/batch'
        xmlns:gd='http://schemas.google.com/g/2005'
        gd:etag='feedEtag'>
      <id>userEmail</id>
      <updated>2008-12-10T10:04:15.446Z</updated>
      <category scheme='http://schemas.google.com/g/2005#kind' term='http://schemas.google.com/contact/2008#contact'/>
      <title>User's Contacts</title>
      <!-- Other entries ... -->
    </feed>

Let’s start to connect to Google Play

Let me summarized the detail step here:

  • Your Android app provide IAP UI for uesr to purchase.
  • User complete purchase process, Google Play will return the receipt to your App.
  • App collect the receipt to server for server certification.
  • Server using Google OAuth2 to connect to Google Play API
  • Request related receipt information and make sure user already purchase.
  • Add some IAP item in server side.

Start from your Android App

In Android Doc: Implementing In-app Billing-Purchasing an Item, onActivityResult get JSON data which contains purchase info as follow:

  
 @Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) { 
   if (requestCode == 1001) {           
      int responseCode = data.getIntExtra("RESPONSE_CODE", 0);
      String purchaseData = data.getStringExtra("INAPP_PURCHASE_DATA");
      String dataSignature = data.getStringExtra("INAPP_DATA_SIGNATURE");//this is the signature which you want

      if (resultCode == RESULT_OK) {
         try {
            JSONObject jo = new JSONObject(purchaseData);//this is the JSONObject which you have included in Your Question right now
            String sku = jo.getString("productId");
            alert("You have bought the " + sku + ". Excellent choice, 
               adventurer!");
          }
          catch (JSONException e) {
             alert("Failed to parse purchase data.");
             e.printStackTrace();
          }
      }
   }
}

Mobile device will provide their IAP receipt result as spec.

    [
      {
        INAPP_PURCHASE_DATA: {
          "orderId": "123456",
          "packageName": "com.example.iap", //Use it later
          "productId": "test",              //Use it later 
          "purchaseTime": 1424588354666,
          "purchaseState": 0,
          "developerPayload": "TEST_DEV_PAYLOAD",
          "purchaseToken": "TOKEN"          //Use it later
        },
        INAPP_DATA_SIGNATURE: "SIGNATURE_CODE",
        RESPONSE_CODE: 0
      }
    ]

Send your request to Google Play

All related IAP API will separate into following:

We will focus on purchase/product first, here is a sample code to connect to Google Play. Please refer here for each parameter your need.

Note: You still need the json file which contains keys and OAuth2 information.

package main

import (
	"encoding/json"
	"golang.org/x/oauth2"
	"golang.org/x/oauth2/google"
	"io/ioutil"
	"log"
	"strconv"
	"time"
)

type GoogleIAP struct {
	Kind               string `json:"kind"`
	PurchaseTimeMillis string `json:"purchaseTimeMillis"`
	PurchaseState      string `json:"purchaseState"`
	ConsumptionState   bool   `json:"consumptionState"`
	DeveloperPayload   string `json:"developerPayload"`
}

func main() {
	data, err := ioutil.ReadFile("downloaded.json")
	if err != nil {
		log.Fatal(err)
	}
	conf, err := google.JWTConfigFromJSON(data, "https://www.google.com/m8/androidpublisher")
	if err != nil {
		log.Fatal(err)
	}

	client := conf.Client(oauth2.NoContext)
	res, err := client.Get("https://www.googleapis.com/androidpublisher/v2/applications/`com.example.iap`/purchases/products/`test`/tokens/`TOKEN`")
	if err != nil {
		log.Println("err:", err)
	}
	body, err := ioutil.ReadAll(res.Body)
	log.Println(string(body))

	appResult := &GoogleIAP{}

	err = json.Unmarshal(body, &appResult)
	log.Printf("Receipt return %+v \n", appResult)

	//To transfer to purchase time millisecond to time.Time
	time_duration, _ := strconv.ParseInt(appResult.PurchaseTimeMillis, 10, 64)
	log.Println(time_duration)

	time_purchase := time.Unix(time_duration/1000, 0)
	log.Println(time_purchase.Local())

	// Compare with receipt to make sure the receipt if valid.

	// 1. appResult.PurchaseTimeMillis need equal to purchaseTime get from App.
	// 2. appResult.ConsumptionState need to be 1. (User already consumed)
	// 3. appResult.PurchaseState need to be 0. (Order is completed.)

	Do_SOMETHING_IN_SERVER_PART()
}

Reference

[Golang] 關於Golang Time 的處理

image

[Golang] 關於Golang Time 的處理

花了一點時間在處理timestamp.工作不外乎是儲存timestamp,解析timestamp並且比對timestamp. 找了很久的相關資料,卻發現處理上並沒有那麼直覺主要是在於以下幾個部分.

####格式: 永遠會困在格式之中

首先最重要的就是處理time的格式,根據Golang Time Pkg,裡面的格式可以有ANSI, RFC822, RFC1123…. 許多種,這裡面大家所熟知的ISO 8601其實沒有名列在其中.但是你可以使用它其中一個profile 那麼就是RFC 3339

儲存格式決定你要如何存取時間的資料.

當然…. 如果大家都是丟time.Time 的格式.. 那麼就世界太平…. 如果不行.. 大家談一下如何parse..

####時區: 時間比對的重要參數

大家都知道,時區是一個濫觴.但是只要能夠正確的處理其實Golang裡面內建已經把時區處理好了.主要用來解析的是ParseParseInLocation

會需要使用到時區的時候,通常是在於接收資料從client或是其他的程式. 這時候比較建議的是:

  • 統一使用UTC 時區,作為字串解析會統一點
  • LoadLocation 是個好東西,可以幫助你轉換時區.前提是你已經拿到time.Time

以下為一個簡單範例去取得UTC的時間

    func GetUTCTime() time.Time {
    	t := time.Now()
    	local_location, err := time.LoadLocation("UTC")
    	if err != nil {
    		fmt.Println(err)
    	}
    	time_UTC := t.In(local_location)
    	return time_UTC
    }

如果你必須要parse一個time.Time不支援的格式,來轉換成time.Time格式.以下是一個簡單的範例.(當然前提資料是 UTC :) )

    func ParseCustomTimeString(time_string string) time.Time {
    	//"2015-03-10 23:58:23 UTC"
    	str_array := strings.Split(time_string, "UTC")
    	var year, mon, day, hh, mm, ss int
    	fmt.Sscanf(str_array[0], "%d-%d-%d %d:%d:%d ", &year, &mon, &day, &hh, &mm, &ss)
    	time_string_to_parse := fmt.Sprintf("%d-%02d-%02dT%02d:%02d:%02d+00:00", year, mon, day, hh, mm, ss)
    	url_create_time, _ := time.Parse(time.RFC3339, time_string_to_parse)
    	return url_create_time
    }

####時間的比對,善用Before/After 針對於時間的操作上面,其實Golang Time Pkg 提供了很方便的比較時間先後的方式. Before/After

其實只要透過以下的方式,就可以比較不同時區,不同時間格式的先後順序.

    func IsWarrantyExpired(time_purchase time.Time) bool {
    	//time of product purchase store in DB which is time.time with ETC
    	time_Warranty := time_purchase.AddDate(1, 0, 0) //假設是一年保固
    	return time_Warranty.After(time.Now())          //如果保固期比現在還後面,代表還是保固期內.
    }

範例程式中, time.Time.AddDate() 是一個可以取出變更過的時間.比如說你需要取出現在這個時間一年後的timestamp,就可以透過這個.目前支援年,月,日. 如果要小時的話,可能就得要手動去改.

####在資料庫裡處理time.Time

其實不論是 MySQL或是 Mongodb都是支援 time_t的.只是要如何透過Golang把 time.Time放入資料庫.這裡有簡單的整理.Go-MySQL-Driver 本身已經支援time.Time. 接下來會介紹如何在MongoDB裡面使用time.Time.

如果是MySQL 可以直接把time.Time 轉成[]byte 來操作,方式如下:

    //MySQL using "[]byte" to handle time_stamp
    //Refer time-stamp in golang
    func (t *Timestamp) MarshalJSON() ([]byte, error) {
    	ts := time.Time(*t).Unix()
    	stamp := fmt.Sprint(ts)
 
    	return []byte(stamp), nil
    }
 
    func (t *Timestamp) UnmarshalJSON(b []byte) error {
    	ts, err := strconv.Atoi(string(b))
    	if err != nil {
    		return err
    	}

如果是MongoDB 則可以直接操作.在MongoDB裡面會把欄位設定成ISODate()

type Person struct {
	ID        bson.ObjectId `bson:"_id,omitempty"`
	Name      string
	Phone     string
	Timestamp time.Time
}

func main() {
	session, err := mgo.Dial("127.0.0.1")
	if err != nil {
		panic(err)
	}

defer session.Close()

// Collection People
c := session.DB("DB").C("test")

// Insert Datas
err = c.Insert(&Person{Name: "fff", Phone: "+55 53 1234 4321", Timestamp: time.Now().AddDate(0, 0, -3)},
	&Person{Name: "gggg", Phone: "+66 33 1234 5678", Timestamp: time.Now().AddDate(0, 0, 4)})

if err != nil {
	panic(err)
}

// Query All and order by timestamp descending
	var results []Person
	err = c.Find(bson.M{"name": bson.M{"$exists": true}}).Sort("-timestamp").All(&results)

if err != nil {
	panic(err)
}
fmt.Println("Results All: ")
for _, v := range results {
	fmt.Println(v.Name, ":", v.Timestamp.Format("2006-01-02T15:04:05Z07:00"))
}

詳細的程式碼,可以參考這裡

相關資料

[XMPP][Ejabberd] Some XMPP spec survey

image

Preface:

This article is a note about I study XMPP spec recent days. To maniplate a XMPP client, it might be easy to deal it with IP*WORKs 3rd party XMPP client. But it could not fulfil some custom request such as:

  • Not response friend request immediatelly, once we got it.
  • Handle roster (friend list) programmatically
  • Maniplate VCard as custom information storage.

In those case, we might need to handle XMPP XML commands to

Ejabberd

image

Ejabberd is a XMPP server(which twitter use it at first). It written by Erlang, it is powerful to handle multiple user connectivity.

XMPP Command

Both we could have two ways to manipulate XMPP, one by 3rd party XMPP framework. We will use IP*Work for a example which you can find sample code from web here.

Another one is using basic XML send to XMPP server directly. Actually we still use IP*Works. sendCommand to send our XML to XMPP server. However, use “SendCommand” could be more powerful and straintforward because we handle all XMPP commands directly.

Subscription Management

Add Subscription (Friend Request)

XMPP module should provide transfer XML command directly to server to do following path:

You can use IQ message from server to filter what you need.

Handle Subscription (Friend Request)

Once you got the subscription request from server, normally you have two way to handle it.

  • If you use some XMPP framework (ex: IP*Works), you can monitor SubscriptionRequest.
    • Note: It will need your force response “accept” or “reject” once you receipt this event. If you want to handle it in lower command set. Refer second way.
  • Using low-level XML command from IQ event.

      //Receive Friend request.
      <iq from='[email protected]/balcony' type='get' id='roster_1'>
          <query xmlns='jabber:iq:roster'/>
      </iq>
    
      //accept
      <presence to='[email protected]' type='subscribed'/>
      //reject
      <presence to='[email protected]' type='unsubscribed'/>
    

Normally, you will need send subscription back to make sure the status subscription. (a.k.a friendship) is bi-direction.

vCard Management

According to XEP:00544, vCard is an existing and widely-used standard for personal user information storage, somewhat like an electronic business card.

It using the XML format which can store any information you need it. Detail spec refer to RFC 2426

The vCard information might present information as follow

How we use vCard?

Because vCard could be any format of XML table.

    //Abstract information from vCard
    <vCard xmlns='vcard-temp'>
        <FN>Peter Saint-Andre</FN>
        <N>
          <FAMILY>Saint-Andre</FAMILY>
          <GIVEN>Peter</GIVEN>
          <MIDDLE/>
        </N>
        <NICKNAME>stpeter</NICKNAME>
        <URL>http://www.xmpp.org/xsf/people/stpeter.shtml</URL>
        <BDAY>1966-08-06</BDAY>
        <ORG>
          <ORGNAME>XMPP Standards Foundation</ORGNAME>
          <ORGUNIT/>
        </ORG>
        <TITLE>Executive Director</TITLE>
        <ROLE>Patron Saint</ROLE>
        <TEL><WORK/><VOICE/><NUMBER>303-308-3282</NUMBER></TEL>
        <ADR>
          <WORK/>
          <EXTADD>Suite 600</EXTADD>
          <STREET>1899 Wynkoop Street</STREET>
          <LOCALITY>Denver</LOCALITY>
          <REGION>CO</REGION>
          <PCODE>80202</PCODE>
          <CTRY>USA</CTRY>
        </ADR>
        <TEL><HOME/><VOICE/><NUMBER>303-555-1212</NUMBER></TEL>
        <JABBERID>[email protected]</JABBERID>
        <DESC>
          More information about me is located on my 
          personal website: http://www.saint-andre.com/
        </DESC>
      </vCard>

As you could observe, the vCard could store address, name, phone number… etc.

So, I am wondering if we could store some custom data in vCard for our programming usage. But here is something you should note before use it.

  • vCard store as combination text:
    • vCard is store as a full text in the database, that’s mean you could not able search it via specific filed.
  • vCard is public:
    • Although vCard only modify by its owner, but it could find by everyone.

Enable vCard management in Ejabberd

Refer to Ejabberd extension. It has following API to management vCard as follow:

      //Get vCard
      vcard-get user host data [data2]
      
      //Set vCard
       vcard-set user host data [data2] content

[Note:]This command is old, need check latest one if you use Ejabberd 2.1 or newer version.

You can following this instructions to enable mod_admin_extra.

Management vCard data

Because there is no specific API in IP*Works to manipulate vCard, so we list it as XML commands.

Here is the detail:

    //Get vCard
    <iq from='[email protected]/roundabout'
        id='v1'
        type='get'>
      <vCard xmlns='vcard-temp'/>
    </iq>

    //Update vCard
    <iq id='v2' type='set'>
      <vCard xmlns='vcard-temp'>
        <FN>Peter Saint-Andre</FN>
      </vCard>
    </iq>    


    //Get other's vCard
    <iq from='[email protected]/roundabout'
        id='v3'
        to='[email protected]'
        type='get'>
      <vCard xmlns='vcard-temp'/>
    </iq>

Conclusion about vCard usage

Because of the result of those studies, the vCard might be a way to store some extra data in your custom IM system. But I will not suggest you to store some information as follow:

  • Information need to be searched by program.
    • vCard information can not search directly. You need parse it via XML tag.
  • Sensitive information
    • It is not suggest that you store some sensitive information such as password or paid information, because everyone can use XMPP protocol to access your vCard info.

Reference Spec/RFCs

[Golang][程式設計週記].. 2015第九週

##雜七雜八感言:

開始把一些內容分成別篇文章,最近多花了很多時間讀XMPP Spec挺有趣的.

##筆記:

[Golang] 關於Golang 有趣的網頁與小工具

[Golang] Meetup

  • Youtube Playlist: London Go Gathering February 2015
  • FOSDEM 2015 Video list
    • Go at CoreOS by +Kelsey Hightower​​
      • This session will discuss using Go to build products that make distributed computing as stress-free as installing a Linux distribution.
    • Finding Bad Needles in Worldwide Haystacks by Dmitry Savintsev
      • Experience of using Go for a large-scale web security scanner
    • Moving MongoDB components to Go by +Norberto Leite​​
      • We love Go and this train is unstoppable!
    • CockroachDB by +Tobias Schottdorf​​
      • Towards an Open-Source Spanner
    • HTTP/2 for Go by +Brad Fitzpatrick​​
      • Overview of HTTP/2 and the design of Go’s support for it
    • Go and the modern enterprise by +Peter Bourgon​​
    • bleve - text indexing for Go by +Marty Schoch​​
    • The State of Go by +Andrew Gerrand​​
    • Go Lightning Talks by +Andrew Gerrand​​
      • The Go community on Go
  • Youtube: FOSDEM 2015 Go present playlist
    • 關於Golng 有關的session集合,總而言之還是HTTP/2跟Go 1.5

[Golang]關於Go的Constant的型別轉換(Type Cast)

image

前言:

原本的問題是出現在Stackoverflow,有人發問template.HTML(“test”) == “test” 可以過,但是template.HTML(“test”) == some_str 卻不能成 功compiler.

原因:

主要的原因是由於Go 本身是static typing的程式語言,而constant本身的型別是未定,並且對於constant在compiler的狀態下是會有一些型別轉換的.

    // 型別尚未確定
    const hello_string = "Hello, 世界"
    
    // 型別確定為字串
    fmt.Printf(" type is %T \n",  hello_string)
    //type is string

範例程式碼:

根據The Go Blog: Constants 裡面有詳細提到關於Go 是 statical typing的一些問題. 接下來用幾個範例解釋:

  
package main

import (
	"fmt"
)

func main() {
	// hello Type is not define for now.
	const hello = "Hello, 世界"
	// Type is String
	const typedHello string = "Hello, 世界"

	// hello type is cast to string
	fmt.Printf("Type of hello is=%T, typedHello is=%T \n", hello, typedHello)
	// hello is non-type it convert to string and compare
	fmt.Println(hello == typedHello)

	//Another sample. Define MyString as a string type
	type MyString string
	var m MyString
	// m is non-type for now.
	m = "Hello, 世界"

	// It will introduce error, because m type is MyString and typeHello is string.
	//fmt.Println(m == typedHello)
	// hello is non-type until compare with m. It will define as MyString type.
	fmt.Println(m == hello)

}

Go Play is here.