request peer info token wrong

README-ZH.md

@@ -23,7 +23,7 @@
 ### 5. 跨平台
 因为轻量，所以很容易支持各个平台。支持主流的操作系统：Windows,Linux,MacOS；和主流的cpu架构：386、amd64、arm、arm64、mipsle、mipsle64、mips、mips64
 ### 6. 高效
-P2P直连可以让你的设备跑满带宽。不论你的设备在任何网络环境，无论NAT1-4（Cone或Symmetric），都支持。依靠Quic协议优秀的拥塞算法，能在糟糕的网络环境获得高带宽低延时。
+P2P直连可以让你的设备跑满带宽。不论你的设备在任何网络环境，无论NAT1-4（Cone或Symmetric），UDP或TCP打洞,UPNP,IPv6都支持。依靠Quic协议优秀的拥塞算法，能在糟糕的网络环境获得高带宽低延时。
 
 ### 7. 二次开发
 基于OpenP2P只需数行代码，就能让原来只能局域网通信的程序，变成任何内网都能通信
@@ -39,8 +39,8 @@ P2P直连可以让你的设备跑满带宽。不论你的设备在任何网络
 分别在本地和远程电脑下载后双击运行，一键安装
 
    ![image](/doc/images/install.png)
-
-   Windows默认会阻止没有花钱买它家证书签名过的程序，选择“仍要运行”即可。
+   
+Windows默认会阻止没有花钱买它家证书签名过的程序，选择“仍要运行”即可。
 
    ![image](/doc/images/win10warn.png)
 
@@ -95,6 +95,7 @@ P2P直连可以让你的设备跑满带宽。不论你的设备在任何网络
 服务端有个调度模型，根据带宽、ping值、稳定性、服务时长，尽可能地使共享节点均匀地提供服务。连接共享节点使用TOTP密码，hmac-sha256算法校验，它是一次性密码，和我们平时使用的手机验证码或银行密码器一样的原理。
 
 ## 编译
+go version go1.18.1+  
 cd到代码根目录，执行
 ```
 export GOPROXY=https://goproxy.io,direct
@@ -104,16 +105,16 @@ go build
 
 ## TODO
 近期计划：
-1. 支持IPv6
-2. 支持随系统自动启动，安装成系统服务
-3. 提供一些免费服务器给特别差的网络，如广电网络
-4. 建立网站，用户可以在网站管理所有P2PApp和设备。查看设备在线状态，升级，增删查改重启P2PApp等
+1. 支持IPv6(100%)
+2. 支持随系统自动启动，安装成系统服务(100%)
+3. 提供一些免费服务器给特别差的网络，如广电网络(100%)
+4. 建立网站，用户可以在网站管理所有P2PApp和设备。查看设备在线状态，升级，增删查改重启P2PApp等(100%)
 5. 建立公众号，用户可在微信公众号管理所有P2PApp和设备
 6. 客户端提供WebUI
 7. 支持自有服务器高并发连接
 8. 共享节点调度模型优化，对不同的运营商优化
 9. 方便二次开发，提供API和lib
-10. 应用层支持UDP协议，实现很简单，但UDP应用较少暂不急
+10. 应用层支持UDP协议，实现很简单，但UDP应用较少暂不急(100%)
 11. 底层通信支持KCP协议，目前仅支持Quic；KCP专门对延时优化，被游戏加速器广泛使用，可以牺牲一定的带宽降低延时
 12. 支持Android系统，让旧手机焕发青春变成移动网关
 13. 支持Windows网上邻居共享文件
@@ -125,8 +126,7 @@ go build
 
 ## 参与贡献
 TODO或ISSUE里如果有你擅长的领域，或者你有特别好的主意，可以加入OpenP2P项目，贡献你的代码。待项目茁壮成长后，你们就是知名开源项目的主要代码贡献者，岂不快哉。
-## 商业合作
-它是一个中国人发起的项目，更懂国内网络环境，更懂用户需求，更好的企业级支持
+
 ## 技术交流
 QQ群：16947733  
 邮箱：openp2p.cn@gmail.com tenderiron@139.com  

README.md

@@ -25,7 +25,7 @@ The code is open source, the P2P tunnel uses TLS1.3+AES double encryption, and t
 Benefit from lightweight, it easily supports most of major OS, like Windows, Linux, MacOS, also most of CPU architecture, like 386、amd64、arm、arm64、mipsle、mipsle64、mips、mips64.
 
 ### 6. Efficient
-P2P direct connection lets your devices make good use of bandwidth.  Your device can be connected in any network environments, even supports NAT1-4 (Cone or Symmetric). Relying on the excellent congestion algorithm of the Quic protocol, high bandwidth and low latency can be obtained in a bad network environment.
+P2P direct connection lets your devices make good use of bandwidth.  Your device can be connected in any network environments, even supports NAT1-4 (Cone or Symmetric),UDP or TCP punching,UPNP,IPv6.  Relying on the excellent congestion algorithm of the Quic protocol, high bandwidth and low latency can be obtained in a bad network environment.
 
 ### 7. Integration
 Your applicaiton can call OpenP2P with a few code to make any internal networks communicate with each other.
@@ -43,11 +43,12 @@ Download on local and remote computers and double-click to run, one-click instal
 
    ![image](/doc/images/install_en.png)
 
-   By default, Windows will block programs that have not been signed by the Microsoft's certificate, and you can select "Run anyway".
+By default, Windows will block programs that have not been signed by the Microsoft's certificate, and you can select "Run anyway".
 
    ![image](/doc/images/win10warn_en.png)
 
    ![image](/doc/images/stillrun_en.png)
+   
 ### 3.New P2PApp
 
 ![image](/doc/images/devices_en.png)
@@ -102,6 +103,7 @@ That's right, the relay node is naturally an man-in-middle, so AES encryption is
 The server side has a scheduling model, which calculate  bandwith, ping value,stability and service duration to provide a well-proportioned service to every share node. It uses TOTP(Time-based One-time Password) with hmac-sha256 algorithem, its theory as same as the cellphone validation code or bank cipher coder.
 
 ## Build
+go version go1.18.1+  
 cd root directory of the socure code and execute
 ```
 export GOPROXY=https://goproxy.io,direct
@@ -111,18 +113,18 @@ go build
 
 ## TODO
 Short-Term:
-1. Support IPv6.
-2. Support auto run when system boot, setup system service.
-3. Provide free servers to some low-performance network.
-4. Build website, users can manage all P2PApp and devices via it. View devices' online status, upgrade, restart or CURD P2PApp .
+1. Support IPv6.(100%)
+2. Support auto run when system boot, setup system service.(100%)
+3. Provide free servers to some low-performance network.(100%)
+4. Build website, users can manage all P2PApp and devices via it. View devices' online status, upgrade, restart or CURD P2PApp .(100%)
 5. Provide wechat official account, user can manage P2PApp nodes and deivce as same as website.
 6. Provide WebUI on client side.
 7. Support high concurrency on server side.
 8. Optimize our share scheduling model for different network operators.
 9. Provide REST APIs and libary for secondary development.
-10. Support UDP at application layer, it is easy to implement but not urgent due to only a few applicaitons using UDP protocol.
+10. Support UDP at application layer, it is easy to implement but not urgent due to only a few applicaitons using UDP protocol.(100%)
 11. Support KCP protocol underlay, currently support Quic only. KCP focus on delay optimization,which has been widely used as game accelerator,it can sacrifice part of bandwidth to reduce timelag. 
-12. Support Android platform, let the phones to be mobile gateway .
+12. Support Android platform, let the phones to be mobile gateway.
 13. Support SMB Windows neighborhood.
 14. Direct connection on intranet, for testing.
 

USAGE-ZH.md

@@ -17,6 +17,12 @@
 >* -sharebandwidth: 作为共享节点时提供带宽，默认10mbps. 如果是光纤大带宽，设置越大效果越好. 0表示不共享，该节点只在私有的P2P网络使用。不加入共享的P2P网络，这样也意味着无法使用别人的共享节点
 >* -loglevel: 需要查看更多调试日志，设置0；默认是1
 
+### 在docker容器里运行openp2p
+我们暂时还没提供官方docker镜像，你可以在随便一个容器里运行
+```
+nohup ./openp2p -d -node OFFICEPC1 -token TOKEN  &
+#这里由于一般的镜像都精简过，install系统服务会失败，所以使用直接daemon模式后台运行
+```
 ## 连接
 ```
 ./openp2p -d -node HOMEPC123 -token TOKEN -appname OfficeWindowsRemote -peernode OFFICEPC1 -dstip 127.0.0.1 -dstport 3389 -srcport 23389
@@ -71,7 +77,7 @@
 # update local client
 ./openp2p update  
 # update remote client
-curl --insecure 'https://openp2p.cn:27182/api/v1/device/YOUR-NODE-NAME/update?user=&password='
+curl --insecure 'https://api.openp2p.cn:27183/api/v1/device/YOUR-NODE-NAME/update?user=&password='
 ```
 
 Windows系统需要设置防火墙放行本程序，程序会自动设置，如果设置失败会影响连接功能。
@@ -85,4 +91,9 @@ firewall-cmd --state
 ## 卸载
 ```
 ./openp2p uninstall
+# 已安装时
+# windows
+C:\Program Files\OpenP2P\openp2p.exe uninstall
+# linux,macos
+sudo /usr/local/openp2p/openp2p uninstall
 ```

USAGE.md

@@ -19,6 +19,13 @@ Or
 >* -sharebandwidth: Provides bandwidth when used as a shared node, the default is 10mbps. If it is a large bandwidth of optical fiber, the larger the setting, the better the effect. 0 means not shared, the node is only used in a private P2P network. Do not join the shared P2P network, which also means that you CAN NOT use other people’s shared nodes
 >* -loglevel: Need to view more debug logs, set 0; the default is 1
 
+### Run in Docker container
+We don't provide official docker image yet, you can run it in any container
+```
+nohup ./openp2p -d -node OFFICEPC1 -token TOKEN  &
+# Since many docker images have been simplified, the install system service will fail, so the daemon mode is used to run in the background
+```
+
 ## Connect
 ```
 ./openp2p -d -node HOMEPC123 -token TOKEN -appname OfficeWindowsRemote -peernode OFFICEPC1 -dstip 127.0.0.1 -dstport 3389 -srcport 23389
@@ -72,7 +79,7 @@ Configuration example
 # update local client
 ./openp2p update  
 # update remote client
-curl --insecure 'https://openp2p.cn:27182/api/v1/device/YOUR-NODE-NAME/update?user=&password='
+curl --insecure 'https://api.openp2p.cn:27183/api/v1/device/YOUR-NODE-NAME/update?user=&password='
 ```
 
 Windows system needs to set up firewall for this program, the program will automatically set the firewall, if the setting fails, the UDP punching will be affected.  
@@ -86,4 +93,9 @@ firewall-cmd --state
 ## Uninstall
 ```
 ./openp2p uninstall
+# when already installed
+# windows
+C:\Program Files\OpenP2P\openp2p.exe uninstall
+# linux,macos
+sudo /usr/local/openp2p/openp2p uninstall
 ```

common.go

@@ -12,6 +12,8 @@ import (
 	"net/http"
 	"os"
 	"os/exec"
+	"strconv"
+	"strings"
 	"time"
 )
 
@@ -77,6 +79,7 @@ func pkcs7UnPadding(origData []byte, dataLen int) ([]byte, error) {
 	return origData[:(dataLen - unPadLen)], nil
 }
 
+// AES-CBC
 func encryptBytes(key []byte, out, in []byte, plainLen int) ([]byte, error) {
 	if len(key) == 0 {
 		return in[:plainLen], nil
@@ -122,20 +125,20 @@ func netInfo() *NetInfo {
 		client := &http.Client{Transport: tr, Timeout: time.Second * 10}
 		r, err := client.Get("https://ifconfig.co/json")
 		if err != nil {
-			gLog.Println(LevelINFO, "netInfo error:", err)
+			gLog.Println(LvINFO, "netInfo error:", err)
 			continue
 		}
 		defer r.Body.Close()
 		buf := make([]byte, 1024*64)
 		n, err := r.Body.Read(buf)
 		if err != nil {
-			gLog.Println(LevelINFO, "netInfo error:", err)
+			gLog.Println(LvINFO, "netInfo error:", err)
 			continue
 		}
 		rsp := NetInfo{}
 		err = json.Unmarshal(buf[:n], &rsp)
 		if err != nil {
-			gLog.Printf(LevelERROR, "wrong NetInfo:%s", err)
+			gLog.Printf(LvERROR, "wrong NetInfo:%s", err)
 			continue
 		}
 		return &rsp
@@ -158,3 +161,42 @@ func defaultNodeName() string {
 	}
 	return name
 }
+
+const EQUAL int = 0
+const GREATER int = 1
+const LESS int = -1
+
+func compareVersion(v1, v2 string) int {
+	if v1 == v2 {
+		return EQUAL
+	}
+	v1Arr := strings.Split(v1, ".")
+	v2Arr := strings.Split(v2, ".")
+	for i, subVer := range v1Arr {
+		if len(v2Arr) <= i {
+			return GREATER
+		}
+		subv1, _ := strconv.Atoi(subVer)
+		subv2, _ := strconv.Atoi(v2Arr[i])
+		if subv1 > subv2 {
+			return GREATER
+		}
+		if subv1 < subv2 {
+			return LESS
+		}
+	}
+	return LESS
+}
+
+func parseMajorVer(ver string) int {
+	v1Arr := strings.Split(ver, ".")
+	if len(v1Arr) > 0 {
+		n, _ := strconv.ParseInt(v1Arr[0], 10, 32)
+		return int(n)
+	}
+	return 0
+}
+
+func IsIPv6(address string) bool {
+	return strings.Count(address, ":") >= 2
+}

common_test.go

@@ -43,3 +43,54 @@ func TestAESCBC(t *testing.T) {
 func TestNetInfo(t *testing.T) {
 	log.Println(netInfo())
 }
+
+func assertCompareVersion(t *testing.T, v1 string, v2 string, result int) {
+	if compareVersion(v1, v2) != result {
+		t.Errorf("compare version %s %s fail\n", v1, v2)
+	}
+}
+func assertParseMajorVer(t *testing.T, v string, result int) {
+	if parseMajorVer(v) != result {
+		t.Errorf("ParseMajorVer %s fail\n", v)
+	}
+}
+func TestCompareVersion(t *testing.T) {
+	// test =
+	assertCompareVersion(t, "0.98.0", "0.98.0", EQUAL)
+	assertCompareVersion(t, "0.98", "0.98", EQUAL)
+	assertCompareVersion(t, "1.4.0", "1.4.0", EQUAL)
+	assertCompareVersion(t, "1.5.0", "1.5.0", EQUAL)
+	// test >
+	assertCompareVersion(t, "0.98.0.22345", "0.98.0.12345", GREATER)
+	assertCompareVersion(t, "1.98.0.12345", "0.98", GREATER)
+	assertCompareVersion(t, "10.98.0.12345", "9.98.0.12345", GREATER)
+	assertCompareVersion(t, "1.4.0", "0.98.0.12345", GREATER)
+	assertCompareVersion(t, "1.4", "0.98.0.12345", GREATER)
+	assertCompareVersion(t, "1", "0.98.0.12345", GREATER)
+	// test <
+	assertCompareVersion(t, "0.98.0.12345", "0.98.0.12346", LESS)
+	assertCompareVersion(t, "9.98.0.12345", "10.98.0.12345", LESS)
+	assertCompareVersion(t, "1.4.2", "1.5.0", LESS)
+	assertCompareVersion(t, "", "1.5.0", LESS)
+
+}
+
+func TestParseMajorVer(t *testing.T) {
+
+	assertParseMajorVer(t, "0.98.0", 0)
+	assertParseMajorVer(t, "0.98", 0)
+	assertParseMajorVer(t, "1.4.0", 1)
+	assertParseMajorVer(t, "1.5.0", 1)
+
+	assertParseMajorVer(t, "0.98.0.22345", 0)
+	assertParseMajorVer(t, "1.98.0.12345", 1)
+	assertParseMajorVer(t, "10.98.0.12345", 10)
+	assertParseMajorVer(t, "1.4.0", 1)
+	assertParseMajorVer(t, "1.4", 1)
+	assertParseMajorVer(t, "1", 1)
+	assertParseMajorVer(t, "2", 2)
+	assertParseMajorVer(t, "3", 3)
+	assertParseMajorVer(t, "2.1.0", 2)
+	assertParseMajorVer(t, "3.0.0", 3)
+
+}

config.go

@@ -11,8 +11,6 @@ import (
 
 var gConf Config
 
-const IntValueNotSet int = -99999999
-
 type AppConfig struct {
 	// required
 	AppName  string
@@ -24,22 +22,32 @@ type AppConfig struct {
 	PeerUser string
 	Enabled  int // default:1
 	// runtime info
-	peerToken       uint64
-	peerNatType     int
-	peerIP          string
-	peerConeNatPort int
-	retryNum        int
-	retryTime       time.Time
-	shareBandwidth  int
+	peerVersion      string
+	peerToken        uint64
+	peerNatType      int
+	hasIPv4          int
+	peerIPv6         string
+	hasUPNPorNATPMP  int
+	peerIP           string
+	peerConeNatPort  int
+	retryNum         int
+	retryTime        time.Time
+	nextRetryTime    time.Time
+	shareBandwidth   int
+	errMsg           string
+	connectTime      time.Time
+	fromToken        uint64
+	linkMode         string
+	isUnderlayServer int // TODO: bool?
 }
 
 // TODO: add loglevel, maxlogfilesize
 type Config struct {
-	Network  NetworkConfig `json:"network"`
-	Apps     []AppConfig   `json:"apps"`
-	LogLevel int
-
-	mtx sync.Mutex
+	Network    NetworkConfig `json:"network"`
+	Apps       []*AppConfig  `json:"apps"`
+	LogLevel   int
+	daemonMode bool
+	mtx        sync.Mutex
 }
 
 func (c *Config) switchApp(app AppConfig, enabled int) {
@@ -48,27 +56,29 @@ func (c *Config) switchApp(app AppConfig, enabled int) {
 	for i := 0; i < len(c.Apps); i++ {
 		if c.Apps[i].Protocol == app.Protocol && c.Apps[i].SrcPort == app.SrcPort {
 			c.Apps[i].Enabled = enabled
+			c.Apps[i].retryNum = 0
+			c.Apps[i].nextRetryTime = time.Now()
 			return
 		}
 	}
 }
 
-func (c *Config) add(app AppConfig, force bool) {
+func (c *Config) add(app AppConfig, override bool) {
 	c.mtx.Lock()
 	defer c.mtx.Unlock()
 	if app.SrcPort == 0 || app.DstPort == 0 {
-		gLog.Println(LevelERROR, "invalid app ", app)
+		gLog.Println(LvERROR, "invalid app ", app)
 		return
 	}
-	for i := 0; i < len(c.Apps); i++ {
-		if c.Apps[i].Protocol == app.Protocol && c.Apps[i].SrcPort == app.SrcPort {
-			if force {
-				c.Apps[i] = app
+	if override {
+		for i := 0; i < len(c.Apps); i++ {
+			if c.Apps[i].Protocol == app.Protocol && c.Apps[i].SrcPort == app.SrcPort {
+				c.Apps[i] = &app // override it
+				return
 			}
-			return
 		}
 	}
-	c.Apps = append(c.Apps, app)
+	c.Apps = append(c.Apps, &app)
 }
 
 func (c *Config) delete(app AppConfig) {
@@ -91,14 +101,20 @@ func (c *Config) save() {
 	data, _ := json.MarshalIndent(c, "", "  ")
 	err := ioutil.WriteFile("config.json", data, 0644)
 	if err != nil {
-		gLog.Println(LevelERROR, "save config.json error:", err)
+		gLog.Println(LvERROR, "save config.json error:", err)
 	}
 }
 
+func init() {
+	gConf.LogLevel = 1
+	gConf.Network.ShareBandwidth = 10
+	gConf.Network.ServerHost = "api.openp2p.cn"
+	gConf.Network.ServerPort = WsPort
+
+}
+
 func (c *Config) load() error {
 	c.mtx.Lock()
-	c.LogLevel = IntValueNotSet
-	c.Network.ShareBandwidth = IntValueNotSet
 	defer c.mtx.Unlock()
 	data, err := ioutil.ReadFile("config.json")
 	if err != nil {
@@ -107,48 +123,78 @@ func (c *Config) load() error {
 	}
 	err = json.Unmarshal(data, &c)
 	if err != nil {
-		gLog.Println(LevelERROR, "parse config.json error:", err)
+		gLog.Println(LvERROR, "parse config.json error:", err)
 	}
 	return err
 }
 
+func (c *Config) setToken(token uint64) {
+	c.mtx.Lock()
+	defer c.mtx.Unlock()
+	c.Network.Token = token
+}
+func (c *Config) setUser(user string) {
+	c.mtx.Lock()
+	defer c.mtx.Unlock()
+	c.Network.User = user
+}
+func (c *Config) setNode(node string) {
+	c.mtx.Lock()
+	defer c.mtx.Unlock()
+	c.Network.Node = node
+}
+func (c *Config) setShareBandwidth(bw int) {
+	c.mtx.Lock()
+	defer c.mtx.Unlock()
+	c.Network.ShareBandwidth = bw
+}
+
 type NetworkConfig struct {
 	// local info
-	Token          uint64
-	Node           string
-	User           string
-	localIP        string
-	ipv6           string
-	mac            string
-	os             string
-	publicIP       string
-	natType        int
-	ShareBandwidth int
+	Token           uint64
+	Node            string
+	User            string
+	localIP         string
+	mac             string
+	os              string
+	publicIP        string
+	natType         int
+	hasIPv4         int
+	publicIPv6      string // must lowwer-case not save json
+	hasUPNPorNATPMP int
+	ShareBandwidth  int
 	// server info
 	ServerHost string
 	ServerPort int
 	UDPPort1   int
 	UDPPort2   int
+	TCPPort    int
 }
 
-func parseParams() {
-	serverHost := flag.String("serverhost", "api.openp2p.cn", "server host ")
+func parseParams(subCommand string) {
+	fset := flag.NewFlagSet(subCommand, flag.ExitOnError)
+	serverHost := fset.String("serverhost", "api.openp2p.cn", "server host ")
+	serverPort := fset.Int("serverport", WsPort, "server port ")
 	// serverHost := flag.String("serverhost", "127.0.0.1", "server host ") // for debug
-	node := flag.String("node", "", "node name. 8-31 characters")
-	token := flag.Uint64("token", 0, "token")
-	peerNode := flag.String("peernode", "", "peer node name that you want to connect")
-	dstIP := flag.String("dstip", "127.0.0.1", "destination ip ")
-	dstPort := flag.Int("dstport", 0, "destination port ")
-	srcPort := flag.Int("srcport", 0, "source port ")
-	protocol := flag.String("protocol", "tcp", "tcp or udp")
-	appName := flag.String("appname", "", "app name")
-	flag.Bool("noshare", false, "deprecated. uses -sharebandwidth 0") // Deprecated, rm later
-	shareBandwidth := flag.Int("sharebandwidth", 10, "N mbps share bandwidth limit, private node no limit")
-	flag.Bool("f", false, "deprecated. config file") // Deprecated, rm later
-	daemonMode := flag.Bool("d", false, "daemonMode")
-	flag.Bool("bydaemon", false, "start by daemon") // Deprecated, rm later
-	logLevel := flag.Int("loglevel", 1, "0:debug 1:info 2:warn 3:error")
-	flag.Parse()
+	token := fset.Uint64("token", 0, "token")
+	node := fset.String("node", "", "node name. 8-31 characters. if not set, it will be hostname")
+	peerNode := fset.String("peernode", "", "peer node name that you want to connect")
+	dstIP := fset.String("dstip", "127.0.0.1", "destination ip ")
+	dstPort := fset.Int("dstport", 0, "destination port ")
+	srcPort := fset.Int("srcport", 0, "source port ")
+	tcpPort := fset.Int("tcpport", 0, "tcp port for upnp or publicip")
+	protocol := fset.String("protocol", "tcp", "tcp or udp")
+	appName := fset.String("appname", "", "app name")
+	shareBandwidth := fset.Int("sharebandwidth", 10, "N mbps share bandwidth limit, private network no limit")
+	daemonMode := fset.Bool("d", false, "daemonMode")
+	notVerbose := fset.Bool("nv", false, "not log console")
+	newconfig := fset.Bool("newconfig", false, "not load existing config.json")
+	logLevel := fset.Int("loglevel", 0, "0:info 1:warn 2:error 3:debug")
+	if subCommand == "" { // no subcommand
+		fset.Parse(os.Args[1:])
+	} else {
+		fset.Parse(os.Args[2:])
+	}
 
 	config := AppConfig{Enabled: 1}
 	config.PeerNode = *peerNode
@@ -157,14 +203,16 @@ func parseParams() {
 	config.SrcPort = *srcPort
 	config.Protocol = *protocol
 	config.AppName = *appName
-	gConf.load()
+	if !*newconfig {
+		gConf.load() // load old config. otherwise will clear all apps
+	}
 	if config.SrcPort != 0 {
 		gConf.add(config, true)
 	}
-	gConf.mtx.Lock()
-
+	// gConf.mtx.Lock() // when calling this func it's single-thread no lock
+	gConf.daemonMode = *daemonMode
 	// spec paramters in commandline will always be used
-	flag.Visit(func(f *flag.Flag) {
+	fset.Visit(func(f *flag.Flag) {
 		if f.Name == "sharebandwidth" {
 			gConf.Network.ShareBandwidth = *shareBandwidth
 		}
@@ -177,37 +225,43 @@ func parseParams() {
 		if f.Name == "loglevel" {
 			gConf.LogLevel = *logLevel
 		}
+		if f.Name == "tcpport" {
+			gConf.Network.TCPPort = *tcpPort
+		}
+		if f.Name == "token" {
+			gConf.Network.Token = *token
+		}
 	})
 
 	if gConf.Network.ServerHost == "" {
 		gConf.Network.ServerHost = *serverHost
 	}
-	if gConf.Network.Node == "" {
-		if *node == "" { // config and param's node both empty
-			hostname := defaultNodeName()
-			node = &hostname
+	if *node != "" {
+		if len(*node) < 8 {
+			gLog.Println(LvERROR, ErrNodeTooShort)
+			os.Exit(9)
 		}
 		gConf.Network.Node = *node
+	} else {
+		if gConf.Network.Node == "" { // if node name not set. use os.Hostname
+			gConf.Network.Node = defaultNodeName()
+		}
 	}
-	if *token != 0 {
-		gConf.Network.Token = *token
-	}
-	if gConf.LogLevel == IntValueNotSet {
-		gConf.LogLevel = *logLevel
-	}
-	if gConf.Network.ShareBandwidth == IntValueNotSet {
-		gConf.Network.ShareBandwidth = *shareBandwidth
+	if gConf.Network.TCPPort == 0 {
+		if *tcpPort == 0 {
+			p := int(nodeNameToID(gConf.Network.Node)%15000 + 50000)
+			tcpPort = &p
+		}
+		gConf.Network.TCPPort = *tcpPort
 	}
 
-	gConf.Network.ServerPort = 27183
-	gConf.Network.UDPPort1 = 27182
-	gConf.Network.UDPPort2 = 27183
+	gConf.Network.ServerPort = *serverPort
+	gConf.Network.UDPPort1 = UDPPort1
+	gConf.Network.UDPPort2 = UDPPort2
 	gLog.setLevel(LogLevel(gConf.LogLevel))
-	gConf.mtx.Unlock()
-	gConf.save()
-	if *daemonMode {
-		d := daemon{}
-		d.run()
-		os.Exit(0)
+	if *notVerbose {
+		gLog.setMode(LogFile)
 	}
+	// gConf.mtx.Unlock()
+	gConf.save()
 }

config.json

@@ -1,10 +1,9 @@
 {
     "network": {
         "Node": "YOUR_NODE_NAME",
-        "User": "YOUR_USER_NAME",
-        "Password": "YOUR_PASSWORD",
-        "ServerHost": "openp2p.cn",
-        "ServerPort": 27182,
+        "Token": "YOUR_TOKEN",
+        "ServerHost": "api.openp2p.cn",
+        "ServerPort": 27183,
         "UDPPort1": 27182,
         "UDPPort2": 27183
     },

daemon.go

@@ -1,13 +1,9 @@
 package main
 
 import (
-	"flag"
 	"fmt"
-	"io"
 	"os"
-	"os/exec"
 	"path/filepath"
-	"strings"
 	"time"
 
 	"github.com/kardianos/service"
@@ -19,34 +15,34 @@ type daemon struct {
 }
 
 func (d *daemon) Start(s service.Service) error {
-	gLog.Println(LevelINFO, "daemon start")
+	gLog.Println(LvINFO, "daemon start")
 	return nil
 }
 
 func (d *daemon) Stop(s service.Service) error {
-	gLog.Println(LevelINFO, "service stop")
+	gLog.Println(LvINFO, "service stop")
 	d.running = false
 	if d.proc != nil {
-		gLog.Println(LevelINFO, "stop worker")
+		gLog.Println(LvINFO, "stop worker")
 		d.proc.Kill()
 	}
 	if service.Interactive() {
-		gLog.Println(LevelINFO, "stop daemon")
+		gLog.Println(LvINFO, "stop daemon")
 		os.Exit(0)
 	}
 	return nil
 }
 
 func (d *daemon) run() {
-	gLog.Println(LevelINFO, "daemon run start")
-	defer gLog.Println(LevelINFO, "daemon run end")
+	gLog.Println(LvINFO, "daemon run start")
+	defer gLog.Println(LvINFO, "daemon run end")
 	d.running = true
 	binPath, _ := os.Executable()
 	mydir, err := os.Getwd()
 	if err != nil {
 		fmt.Println(err)
 	}
-	gLog.Println(LevelINFO, mydir)
+	gLog.Println(LvINFO, mydir)
 	conf := &service.Config{
 		Name:        ProducnName,
 		DisplayName: ProducnName,
@@ -64,21 +60,35 @@ func (d *daemon) run() {
 			break
 		}
 	}
+	args = append(args, "-nv")
 	for {
 		// start worker
-		gLog.Println(LevelINFO, "start worker process, args:", args)
-		execSpec := &os.ProcAttr{Files: []*os.File{os.Stdin, os.Stdout, os.Stderr}}
+		tmpDump := filepath.Join("log", "dump.log.tmp")
+		dumpFile := filepath.Join("log", "dump.log")
+		f, err := os.Create(filepath.Join(tmpDump))
+		if err != nil {
+			gLog.Printf(LvERROR, "start worker error:%s", err)
+			return
+		}
+		gLog.Println(LvINFO, "start worker process, args:", args)
+		execSpec := &os.ProcAttr{Env: append(os.Environ(), "GOTRACEBACK=crash"), Files: []*os.File{os.Stdin, os.Stdout, f}}
 		p, err := os.StartProcess(binPath, args, execSpec)
 		if err != nil {
-			gLog.Printf(LevelERROR, "start worker error:%s", err)
+			gLog.Printf(LvERROR, "start worker error:%s", err)
 			return
 		}
 		d.proc = p
 		_, _ = p.Wait()
+		f.Close()
+		time.Sleep(time.Second)
+		err = os.Rename(tmpDump, dumpFile)
+		if err != nil {
+			gLog.Printf(LvERROR, "rename dump error:%s", err)
+		}
 		if !d.running {
 			return
 		}
-		gLog.Printf(LevelERROR, "worker stop, restart it after 10s")
+		gLog.Printf(LvERROR, "worker stop, restart it after 10s")
 		time.Sleep(time.Second * 10)
 	}
 }
@@ -103,159 +113,3 @@ func (d *daemon) Control(ctrlComm string, exeAbsPath string, args []string) erro
 
 	return nil
 }
-
-// examples:
-// listen:
-// ./openp2p install -node hhd1207-222 -token YOUR-TOKEN -sharebandwidth 0
-// listen and build p2papp:
-// ./openp2p install -node hhd1207-222 -token YOUR-TOKEN -sharebandwidth 0 -peernode hhdhome-n1 -dstip 127.0.0.1 -dstport 50022 -protocol tcp -srcport 22
-func install() {
-	gLog.Println(LevelINFO, "install start")
-	defer gLog.Println(LevelINFO, "install end")
-	// auto uninstall
-
-	uninstall()
-	// save config file
-	installFlag := flag.NewFlagSet("install", flag.ExitOnError)
-	serverHost := installFlag.String("serverhost", "api.openp2p.cn", "server host ")
-	// serverHost := flag.String("serverhost", "127.0.0.1", "server host ") // for debug
-	token := installFlag.Uint64("token", 0, "token")
-	node := installFlag.String("node", "", "node name. 8-31 characters. if not set, it will be hostname")
-	peerNode := installFlag.String("peernode", "", "peer node name that you want to connect")
-	dstIP := installFlag.String("dstip", "127.0.0.1", "destination ip ")
-	dstPort := installFlag.Int("dstport", 0, "destination port ")
-	srcPort := installFlag.Int("srcport", 0, "source port ")
-	protocol := installFlag.String("protocol", "tcp", "tcp or udp")
-	appName := flag.String("appname", "", "app name")
-	installFlag.Bool("noshare", false, "deprecated. uses -sharebandwidth 0")
-	shareBandwidth := installFlag.Int("sharebandwidth", 10, "N mbps share bandwidth limit, private node no limit")
-	logLevel := installFlag.Int("loglevel", 1, "0:debug 1:info 2:warn 3:error")
-	installFlag.Parse(os.Args[2:])
-	if *node != "" && len(*node) < 8 {
-		gLog.Println(LevelERROR, ErrNodeTooShort)
-		os.Exit(9)
-	}
-	if *node == "" { // if node name not set. use os.Hostname
-		hostname := defaultNodeName()
-		node = &hostname
-	}
-	gConf.load() // load old config. otherwise will clear all apps
-	gConf.LogLevel = *logLevel
-	gConf.Network.ServerHost = *serverHost
-	gConf.Network.Token = *token
-	gConf.Network.Node = *node
-	gConf.Network.ServerPort = 27183
-	gConf.Network.UDPPort1 = 27182
-	gConf.Network.UDPPort2 = 27183
-	gConf.Network.ShareBandwidth = *shareBandwidth
-	config := AppConfig{Enabled: 1}
-	config.PeerNode = *peerNode
-	config.DstHost = *dstIP
-	config.DstPort = *dstPort
-	config.SrcPort = *srcPort
-	config.Protocol = *protocol
-	config.AppName = *appName
-	if config.SrcPort != 0 {
-		gConf.add(config, true)
-	}
-	err := os.MkdirAll(defaultInstallPath, 0775)
-	if err != nil {
-		gLog.Printf(LevelERROR, "MkdirAll %s error:%s", defaultInstallPath, err)
-		return
-	}
-	err = os.Chdir(defaultInstallPath)
-	if err != nil {
-		gLog.Println(LevelERROR, "cd error:", err)
-		return
-	}
-	gConf.save()
-	targetPath := filepath.Join(defaultInstallPath, defaultBinName)
-	d := daemon{}
-	// copy files
-
-	binPath, _ := os.Executable()
-	src, errFiles := os.Open(binPath) // can not use args[0], on Windows call openp2p is ok(=openp2p.exe)
-	if errFiles != nil {
-		gLog.Printf(LevelERROR, "os.OpenFile %s error:%s", os.Args[0], errFiles)
-		return
-	}
-
-	dst, errFiles := os.OpenFile(targetPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0775)
-	if errFiles != nil {
-		gLog.Printf(LevelERROR, "os.OpenFile %s error:%s", targetPath, errFiles)
-		return
-	}
-
-	_, errFiles = io.Copy(dst, src)
-	if errFiles != nil {
-		gLog.Printf(LevelERROR, "io.Copy error:%s", errFiles)
-		return
-	}
-	src.Close()
-	dst.Close()
-
-	// install system service
-	// args := []string{""}
-	gLog.Println(LevelINFO, "targetPath:", targetPath)
-	err = d.Control("install", targetPath, []string{"-d"})
-	if err == nil {
-		gLog.Println(LevelINFO, "install system service ok.")
-	}
-	time.Sleep(time.Second * 2)
-	err = d.Control("start", targetPath, []string{"-d"})
-	if err != nil {
-		gLog.Println(LevelERROR, "start openp2p service error:", err)
-	} else {
-		gLog.Println(LevelINFO, "start openp2p service ok.")
-	}
-}
-
-func installByFilename() {
-	params := strings.Split(filepath.Base(os.Args[0]), "-")
-	if len(params) < 4 {
-		return
-	}
-	serverHost := params[1]
-	token := params[2]
-	gLog.Println(LevelINFO, "install start")
-	targetPath := os.Args[0]
-	args := []string{"install"}
-	args = append(args, "-serverhost")
-	args = append(args, serverHost)
-	args = append(args, "-token")
-	args = append(args, token)
-	env := os.Environ()
-	cmd := exec.Command(targetPath, args...)
-	cmd.Stdout = os.Stdout
-	cmd.Stderr = os.Stderr
-	cmd.Stdin = os.Stdin
-	cmd.Env = env
-	err := cmd.Run()
-	if err != nil {
-		gLog.Println(LevelERROR, "install by filename, start process error:", err)
-		return
-	}
-	gLog.Println(LevelINFO, "install end")
-	fmt.Println("Press the Any Key to exit")
-	fmt.Scanln()
-	os.Exit(0)
-}
-func uninstall() {
-	gLog.Println(LevelINFO, "uninstall start")
-	defer gLog.Println(LevelINFO, "uninstall end")
-	d := daemon{}
-	err := d.Control("stop", "", nil)
-	if err != nil { // service maybe not install
-		return
-	}
-	err = d.Control("uninstall", "", nil)
-	if err != nil {
-		gLog.Println(LevelERROR, "uninstall system service error:", err)
-	} else {
-		gLog.Println(LevelINFO, "uninstall system service ok.")
-	}
-	binPath := filepath.Join(defaultInstallPath, defaultBinName)
-	os.Remove(binPath + "0")
-	os.Remove(binPath)
-	// os.RemoveAll(defaultInstallPath)  // reserve config.json
-}

errorcode.go

@@ -8,9 +8,12 @@ import (
 var (
 	// ErrorS2S string = "s2s is not supported"
 	// ErrorHandshake string = "handshake error"
-	ErrorS2S        = errors.New("s2s is not supported")
-	ErrorHandshake  = errors.New("handshake error")
-	ErrorNewUser    = errors.New("new user")
-	ErrorLogin      = errors.New("user or password not correct")
-	ErrNodeTooShort = errors.New("node name too short, it must >=8 charaters")
+	ErrorS2S                = errors.New("s2s is not supported")
+	ErrorHandshake          = errors.New("handshake error")
+	ErrorNewUser            = errors.New("new user")
+	ErrorLogin              = errors.New("user or password not correct")
+	ErrNodeTooShort         = errors.New("node name too short, it must >=8 charaters")
+	ErrPeerOffline          = errors.New("peer offline")
+	ErrMsgFormat            = errors.New("message format wrong")
+	ErrVersionNotCompatible = errors.New("version not compatible")
 )

go.mod

@@ -1,10 +1,28 @@
 module openp2p
 
-go 1.16
+go 1.18
 
 require (
 	github.com/gorilla/websocket v1.4.2
 	github.com/kardianos/service v1.2.0
-	github.com/lucas-clemente/quic-go v0.24.0
-	golang.org/x/sys v0.0.0-20210906170528-6f6e22806c34
+	github.com/lucas-clemente/quic-go v0.27.0
+	github.com/openp2p-cn/go-reuseport v0.3.2
+	golang.org/x/sys v0.0.0-20220422013727-9388b58f7150
+)
+
+require (
+	github.com/cheekybits/genny v1.0.0 // indirect
+	github.com/fsnotify/fsnotify v1.4.9 // indirect
+	github.com/go-task/slim-sprig v0.0.0-20210107165309-348f09dbbbc0 // indirect
+	github.com/marten-seemann/qtls-go1-16 v0.1.5 // indirect
+	github.com/marten-seemann/qtls-go1-17 v0.1.1 // indirect
+	github.com/marten-seemann/qtls-go1-18 v0.1.1 // indirect
+	github.com/nxadm/tail v1.4.8 // indirect
+	github.com/onsi/ginkgo v1.16.4 // indirect
+	golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9 // indirect
+	golang.org/x/mod v0.4.2 // indirect
+	golang.org/x/net v0.0.0-20210428140749-89ef3d95e781 // indirect
+	golang.org/x/tools v0.1.1 // indirect
+	golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
+	gopkg.in/tomb.v1 v1.0.0-20141024135613-dd632973f1e7 // indirect
 )

handlepush.go

@@ -17,37 +17,54 @@ func handlePush(pn *P2PNetwork, subType uint16, msg []byte) error {
 	if err != nil {
 		return err
 	}
-	gLog.Printf(LevelDEBUG, "handle push msg type:%d, push header:%+v", subType, pushHead)
+	gLog.Printf(LvDEBUG, "handle push msg type:%d, push header:%+v", subType, pushHead)
 	switch subType {
-	case MsgPushConnectReq:
+	case MsgPushConnectReq: // TODO: handle a msg move to a new function
 		req := PushConnectReq{}
 		err := json.Unmarshal(msg[openP2PHeaderSize+PushHeaderSize:], &req)
 		if err != nil {
-			gLog.Printf(LevelERROR, "wrong MsgPushConnectReq:%s", err)
+			gLog.Printf(LvERROR, "wrong MsgPushConnectReq:%s", err)
 			return err
 		}
-		gLog.Printf(LevelINFO, "%s is connecting...", req.From)
-		gLog.Println(LevelDEBUG, "push connect response to ", req.From)
+		gLog.Printf(LvINFO, "%s is connecting...", req.From)
+		gLog.Println(LvDEBUG, "push connect response to ", req.From)
+		if compareVersion(req.Version, LeastSupportVersion) == LESS {
+			gLog.Println(LvERROR, ErrVersionNotCompatible.Error(), ":", req.From)
+			rsp := PushConnectRsp{
+				Error:  10,
+				Detail: ErrVersionNotCompatible.Error(),
+				To:     req.From,
+				From:   pn.config.Node,
+			}
+			pn.push(req.From, MsgPushConnectRsp, rsp)
+			return ErrVersionNotCompatible
+		}
 		// verify totp token or token
 		if VerifyTOTP(req.Token, pn.config.Token, time.Now().Unix()+(pn.serverTs-pn.localTs)) || // localTs may behind, auto adjust ts
-			VerifyTOTP(req.Token, pn.config.Token, time.Now().Unix()) ||
-			(req.FromToken == pn.config.Token) {
-			gLog.Printf(LevelINFO, "Access Granted\n")
+			VerifyTOTP(req.Token, pn.config.Token, time.Now().Unix()) {
+			gLog.Printf(LvINFO, "Access Granted\n")
 			config := AppConfig{}
 			config.peerNatType = req.NatType
 			config.peerConeNatPort = req.ConeNatPort
 			config.peerIP = req.FromIP
 			config.PeerNode = req.From
+			config.peerVersion = req.Version
+			config.fromToken = req.Token
+			config.peerIPv6 = req.IPv6
+			config.hasIPv4 = req.HasIPv4
+			config.hasUPNPorNATPMP = req.HasUPNPorNATPMP
+			config.linkMode = req.LinkMode
+			config.isUnderlayServer = req.IsUnderlayServer
 			// share relay node will limit bandwidth
-			if req.FromToken != pn.config.Token {
-				gLog.Printf(LevelINFO, "set share bandwidth %d mbps", pn.config.ShareBandwidth)
+			if req.Token != pn.config.Token {
+				gLog.Printf(LvINFO, "set share bandwidth %d mbps", pn.config.ShareBandwidth)
 				config.shareBandwidth = pn.config.ShareBandwidth
 			}
 			// go pn.AddTunnel(config, req.ID)
 			go pn.addDirectTunnel(config, req.ID)
 			break
 		}
-		gLog.Println(LevelERROR, "Access Denied:", req.From)
+		gLog.Println(LvERROR, "Access Denied:", req.From)
 		rsp := PushConnectRsp{
 			Error:  1,
 			Detail: fmt.Sprintf("connect to %s error: Access Denied", pn.config.Node),
@@ -59,19 +76,19 @@ func handlePush(pn *P2PNetwork, subType uint16, msg []byte) error {
 		rsp := PushRsp{}
 		err := json.Unmarshal(msg[openP2PHeaderSize:], &rsp)
 		if err != nil {
-			gLog.Printf(LevelERROR, "wrong pushRsp:%s", err)
+			gLog.Printf(LvERROR, "wrong pushRsp:%s", err)
 			return err
 		}
 		if rsp.Error == 0 {
-			gLog.Printf(LevelDEBUG, "push ok, detail:%s", rsp.Detail)
+			gLog.Printf(LvDEBUG, "push ok, detail:%s", rsp.Detail)
 		} else {
-			gLog.Printf(LevelERROR, "push error:%d, detail:%s", rsp.Error, rsp.Detail)
+			gLog.Printf(LvERROR, "push error:%d, detail:%s", rsp.Error, rsp.Detail)
 		}
 	case MsgPushAddRelayTunnelReq:
 		req := AddRelayTunnelReq{}
 		err := json.Unmarshal(msg[openP2PHeaderSize+PushHeaderSize:], &req)
 		if err != nil {
-			gLog.Printf(LevelERROR, "wrong RelayNodeRsp:%s", err)
+			gLog.Printf(LvERROR, "wrong RelayNodeRsp:%s", err)
 			return err
 		}
 		config := AppConfig{}
@@ -83,13 +100,20 @@ func handlePush(pn *P2PNetwork, subType uint16, msg []byte) error {
 				// notify peer relay ready
 				msg := TunnelMsg{ID: t.id}
 				pn.push(r.From, MsgPushAddRelayTunnelRsp, msg)
-				SaveKey(req.AppID, req.AppKey)
 			}
 
 		}(req)
+	case MsgPushAPPKey:
+		req := APPKeySync{}
+		err := json.Unmarshal(msg[openP2PHeaderSize+PushHeaderSize:], &req)
+		if err != nil {
+			gLog.Printf(LvERROR, "wrong APPKeySync:%s", err)
+			return err
+		}
+		SaveKey(req.AppID, req.AppKey)
 	case MsgPushUpdate:
-		gLog.Println(LevelINFO, "MsgPushUpdate")
-		update() // download new version first, then exec ./openp2p update
+		gLog.Println(LvINFO, "MsgPushUpdate")
+		update(pn.config.ServerHost, pn.config.ServerPort) // download new version first, then exec ./openp2p update
 		targetPath := filepath.Join(defaultInstallPath, defaultBinName)
 		args := []string{"update"}
 		env := os.Environ()
@@ -104,48 +128,100 @@ func handlePush(pn *P2PNetwork, subType uint16, msg []byte) error {
 		}
 		return err
 	case MsgPushRestart:
-		gLog.Println(LevelINFO, "MsgPushRestart")
+		gLog.Println(LvINFO, "MsgPushRestart")
 		os.Exit(0)
 		return err
 	case MsgPushReportApps:
-		gLog.Println(LevelINFO, "MsgPushReportApps")
+		gLog.Println(LvINFO, "MsgPushReportApps")
 		req := ReportApps{}
-		// TODO: add the retrying apps
 		gConf.mtx.Lock()
 		defer gConf.mtx.Unlock()
 		for _, config := range gConf.Apps {
 			appActive := 0
+			relayNode := ""
+			relayMode := ""
+			linkMode := LinkModeUDPPunch
 			i, ok := pn.apps.Load(fmt.Sprintf("%s%d", config.Protocol, config.SrcPort))
 			if ok {
 				app := i.(*p2pApp)
 				if app.isActive() {
 					appActive = 1
 				}
+				relayNode = app.relayNode
+				relayMode = app.relayMode
+				linkMode = app.tunnel.linkModeWeb
 			}
 			appInfo := AppInfo{
-				AppName:  config.AppName,
-				Protocol: config.Protocol,
-				SrcPort:  config.SrcPort,
-				// RelayNode:   relayNode,
+				AppName:     config.AppName,
+				Error:       config.errMsg,
+				Protocol:    config.Protocol,
+				SrcPort:     config.SrcPort,
+				RelayNode:   relayNode,
+				RelayMode:   relayMode,
+				LinkMode:    linkMode,
 				PeerNode:    config.PeerNode,
 				DstHost:     config.DstHost,
 				DstPort:     config.DstPort,
 				PeerUser:    config.PeerUser,
 				PeerIP:      config.peerIP,
 				PeerNatType: config.peerNatType,
-				RetryTime:   config.retryTime.String(),
+				RetryTime:   config.retryTime.Local().Format("2006-01-02T15:04:05-0700"),
+				ConnectTime: config.connectTime.Local().Format("2006-01-02T15:04:05-0700"),
 				IsActive:    appActive,
 				Enabled:     config.Enabled,
 			}
 			req.Apps = append(req.Apps, appInfo)
 		}
 		pn.write(MsgReport, MsgReportApps, &req)
+	case MsgPushReportLog:
+		gLog.Println(LvINFO, "MsgPushReportLog")
+		req := ReportLogReq{}
+		err := json.Unmarshal(msg[openP2PHeaderSize:], &req)
+		if err != nil {
+			gLog.Printf(LvERROR, "wrong MsgPushReportLog:%s  %s", err, string(msg[openP2PHeaderSize:]))
+			return err
+		}
+		if req.FileName == "" {
+			req.FileName = "openp2p.log"
+		}
+		f, err := os.Open(filepath.Join("log", req.FileName))
+		if err != nil {
+			gLog.Println(LvERROR, "read log file error:", err)
+			break
+		}
+		fi, err := f.Stat()
+		if err != nil {
+			break
+		}
+		if req.Offset == 0 && fi.Size() > 4096 {
+			req.Offset = fi.Size() - 4096
+		}
+		if req.Len <= 0 {
+			req.Len = 4096
+		}
+		f.Seek(req.Offset, 0)
+		if req.Len > 1024*1024 { // too large
+			break
+		}
+		buff := make([]byte, req.Len)
+		readLength, err := f.Read(buff)
+		f.Close()
+		if err != nil {
+			gLog.Println(LvERROR, "read log content error:", err)
+			break
+		}
+		rsp := ReportLogRsp{}
+		rsp.Content = string(buff[:readLength])
+		rsp.FileName = req.FileName
+		rsp.Total = fi.Size()
+		rsp.Len = req.Len
+		pn.write(MsgReport, MsgPushReportLog, &rsp)
 	case MsgPushEditApp:
-		gLog.Println(LevelINFO, "MsgPushEditApp")
+		gLog.Println(LvINFO, "MsgPushEditApp")
 		newApp := AppInfo{}
 		err := json.Unmarshal(msg[openP2PHeaderSize:], &newApp)
 		if err != nil {
-			gLog.Printf(LevelERROR, "wrong MsgPushEditApp:%s  %s", err, string(msg[openP2PHeaderSize:]))
+			gLog.Printf(LvERROR, "wrong MsgPushEditApp:%s  %s", err, string(msg[openP2PHeaderSize:]))
 			return err
 		}
 		oldConf := AppConfig{Enabled: 1}
@@ -169,30 +245,28 @@ func handlePush(pn *P2PNetwork, subType uint16, msg []byte) error {
 		// pn.AddApp(config)
 		// TODO: report result
 	case MsgPushEditNode:
-		gLog.Println(LevelINFO, "MsgPushEditNode")
+		gLog.Println(LvINFO, "MsgPushEditNode")
 		req := EditNode{}
 		err := json.Unmarshal(msg[openP2PHeaderSize:], &req)
 		if err != nil {
-			gLog.Printf(LevelERROR, "wrong MsgPushEditNode:%s  %s", err, string(msg[openP2PHeaderSize:]))
+			gLog.Printf(LvERROR, "wrong MsgPushEditNode:%s  %s", err, string(msg[openP2PHeaderSize:]))
 			return err
 		}
-		gConf.mtx.Lock()
-		gConf.Network.Node = req.NewName
-		gConf.Network.ShareBandwidth = req.Bandwidth
-		gConf.mtx.Unlock()
+		gConf.setNode(req.NewName)
+		gConf.setShareBandwidth(req.Bandwidth)
 		gConf.save()
 		// TODO: hot reload
 		os.Exit(0)
 	case MsgPushSwitchApp:
-		gLog.Println(LevelINFO, "MsgPushSwitchApp")
+		gLog.Println(LvINFO, "MsgPushSwitchApp")
 		app := AppInfo{}
 		err := json.Unmarshal(msg[openP2PHeaderSize:], &app)
 		if err != nil {
-			gLog.Printf(LevelERROR, "wrong MsgPushSwitchApp:%s  %s", err, string(msg[openP2PHeaderSize:]))
+			gLog.Printf(LvERROR, "wrong MsgPushSwitchApp:%s  %s", err, string(msg[openP2PHeaderSize:]))
 			return err
 		}
 		config := AppConfig{Enabled: app.Enabled, SrcPort: app.SrcPort, Protocol: app.Protocol}
-		gLog.Println(LevelINFO, app.AppName, " switch to ", app.Enabled)
+		gLog.Println(LvINFO, app.AppName, " switch to ", app.Enabled)
 		gConf.switchApp(config, app.Enabled)
 		if app.Enabled == 0 {
 			// disable APP

holepunch.go

@@ -11,8 +11,8 @@ import (
 )
 
 func handshakeC2C(t *P2PTunnel) (err error) {
-	gLog.Printf(LevelDEBUG, "handshakeC2C %s:%d:%d to %s:%d", t.pn.config.Node, t.coneLocalPort, t.coneNatPort, t.config.peerIP, t.config.peerConeNatPort)
-	defer gLog.Printf(LevelDEBUG, "handshakeC2C ok")
+	gLog.Printf(LvDEBUG, "handshakeC2C %s:%d:%d to %s:%d", t.pn.config.Node, t.coneLocalPort, t.coneNatPort, t.config.peerIP, t.config.peerConeNatPort)
+	defer gLog.Printf(LvDEBUG, "handshakeC2C end")
 	conn, err := net.ListenUDP("udp", t.la)
 	if err != nil {
 		return err
@@ -20,49 +20,51 @@ func handshakeC2C(t *P2PTunnel) (err error) {
 	defer conn.Close()
 	_, err = UDPWrite(conn, t.ra, MsgP2P, MsgPunchHandshake, P2PHandshakeReq{ID: t.id})
 	if err != nil {
-		gLog.Println(LevelDEBUG, "handshakeC2C write MsgPunchHandshake error:", err)
+		gLog.Println(LvDEBUG, "handshakeC2C write MsgPunchHandshake error:", err)
 		return err
 	}
 	ra, head, _, _, err := UDPRead(conn, 5000)
 	if err != nil {
 		time.Sleep(time.Millisecond * 200)
-		gLog.Println(LevelDEBUG, err, ", return this error when ip was not reachable, retry read")
+		gLog.Println(LvDEBUG, err, ", return this error when ip was not reachable, retry read")
 		ra, head, _, _, err = UDPRead(conn, 5000)
 		if err != nil {
-			gLog.Println(LevelDEBUG, "handshakeC2C read MsgPunchHandshake error:", err)
+			gLog.Println(LvDEBUG, "handshakeC2C read MsgPunchHandshake error:", err)
 			return err
 		}
 	}
 	t.ra, _ = net.ResolveUDPAddr("udp", ra.String())
 	// cone server side
 	if head.MainType == MsgP2P && head.SubType == MsgPunchHandshake {
-		gLog.Printf(LevelDEBUG, "read %d handshake ", t.id)
+		gLog.Printf(LvDEBUG, "read %d handshake ", t.id)
 		UDPWrite(conn, t.ra, MsgP2P, MsgPunchHandshakeAck, P2PHandshakeReq{ID: t.id})
 		_, head, _, _, err = UDPRead(conn, 5000)
 		if err != nil {
-			gLog.Println(LevelDEBUG, "handshakeC2C write MsgPunchHandshakeAck error", err)
+			gLog.Println(LvDEBUG, "handshakeC2C write MsgPunchHandshakeAck error", err)
 			return err
 		}
 		if head.MainType == MsgP2P && head.SubType == MsgPunchHandshakeAck {
-			gLog.Printf(LevelDEBUG, "read %d handshake ack ", t.id)
+			gLog.Printf(LvDEBUG, "read %d handshake ack ", t.id)
+			gLog.Printf(LvINFO, "handshakeC2C ok")
 			return nil
 		}
 	}
 	// cone client side will only read handshake ack
 	if head.MainType == MsgP2P && head.SubType == MsgPunchHandshakeAck {
-		gLog.Printf(LevelDEBUG, "read %d handshake ack ", t.id)
+		gLog.Printf(LvDEBUG, "read %d handshake ack ", t.id)
 		_, err = UDPWrite(conn, t.ra, MsgP2P, MsgPunchHandshakeAck, P2PHandshakeReq{ID: t.id})
 		if err != nil {
-			gLog.Println(LevelDEBUG, "handshakeC2C write MsgPunchHandshakeAck error", err)
+			gLog.Println(LvDEBUG, "handshakeC2C write MsgPunchHandshakeAck error", err)
+			return err
 		}
-		return err
 	}
+	gLog.Printf(LvINFO, "handshakeC2C ok")
 	return nil
 }
 
 func handshakeC2S(t *P2PTunnel) error {
-	gLog.Printf(LevelDEBUG, "handshakeC2S start")
-	defer gLog.Printf(LevelDEBUG, "handshakeC2S end")
+	gLog.Printf(LvDEBUG, "handshakeC2S start")
+	defer gLog.Printf(LvDEBUG, "handshakeC2S end")
 	// even if read timeout, continue handshake
 	t.pn.read(t.config.PeerNode, MsgPush, MsgPushHandshakeStart, SymmetricHandshakeAckTimeout)
 	r := rand.New(rand.NewSource(time.Now().UnixNano()))
@@ -73,7 +75,7 @@ func handshakeC2S(t *P2PTunnel) error {
 	}
 	defer conn.Close()
 	go func() error {
-		gLog.Printf(LevelDEBUG, "send symmetric handshake to %s from %d:%d start", t.config.peerIP, t.coneLocalPort, t.coneNatPort)
+		gLog.Printf(LvDEBUG, "send symmetric handshake to %s from %d:%d start", t.config.peerIP, t.coneLocalPort, t.coneNatPort)
 		for i := 0; i < SymmetricHandshakeNum; i++ {
 			// TODO: auto calc cost time
 			time.Sleep(SymmetricHandshakeInterval)
@@ -83,47 +85,48 @@ func handshakeC2S(t *P2PTunnel) error {
 			}
 			_, err = UDPWrite(conn, dst, MsgP2P, MsgPunchHandshake, P2PHandshakeReq{ID: t.id})
 			if err != nil {
-				gLog.Println(LevelDEBUG, "handshakeC2S write MsgPunchHandshake error:", err)
+				gLog.Println(LvDEBUG, "handshakeC2S write MsgPunchHandshake error:", err)
 				return err
 			}
 		}
-		gLog.Println(LevelDEBUG, "send symmetric handshake end")
+		gLog.Println(LvDEBUG, "send symmetric handshake end")
 		return nil
 	}()
 	deadline := time.Now().Add(SymmetricHandshakeAckTimeout)
 	err = conn.SetReadDeadline(deadline)
 	if err != nil {
-		gLog.Println(LevelERROR, "SymmetricHandshakeAckTimeout SetReadDeadline error")
+		gLog.Println(LvERROR, "SymmetricHandshakeAckTimeout SetReadDeadline error")
 		return err
 	}
 	// read response of the punching hole ok port
 	result := make([]byte, 1024)
 	_, dst, err := conn.ReadFrom(result)
 	if err != nil {
-		gLog.Println(LevelERROR, "handshakeC2S wait timeout")
+		gLog.Println(LvERROR, "handshakeC2S wait timeout")
 		return err
 	}
 	head := &openP2PHeader{}
 	err = binary.Read(bytes.NewReader(result[:openP2PHeaderSize]), binary.LittleEndian, head)
 	if err != nil {
-		gLog.Println(LevelERROR, "parse p2pheader error:", err)
+		gLog.Println(LvERROR, "parse p2pheader error:", err)
 		return err
 	}
 	t.ra, _ = net.ResolveUDPAddr("udp", dst.String())
 	if head.MainType == MsgP2P && head.SubType == MsgPunchHandshakeAck {
-		gLog.Printf(LevelDEBUG, "handshakeC2S read %d handshake ack %s", t.id, dst.String())
+		gLog.Printf(LvDEBUG, "handshakeC2S read %d handshake ack %s", t.id, dst.String())
 		_, err = UDPWrite(conn, dst, MsgP2P, MsgPunchHandshakeAck, P2PHandshakeReq{ID: t.id})
 		return err
 	}
+	gLog.Printf(LvINFO, "handshakeC2S ok")
 	return nil
 }
 
 func handshakeS2C(t *P2PTunnel) error {
-	gLog.Printf(LevelDEBUG, "handshakeS2C start")
-	defer gLog.Printf(LevelDEBUG, "handshakeS2C end")
+	gLog.Printf(LvDEBUG, "handshakeS2C start")
+	defer gLog.Printf(LvDEBUG, "handshakeS2C end")
 	gotCh := make(chan *net.UDPAddr, 5)
 	// sequencely udp send handshake, do not parallel send
-	gLog.Printf(LevelDEBUG, "send symmetric handshake to %s:%d start", t.config.peerIP, t.config.peerConeNatPort)
+	gLog.Printf(LvDEBUG, "send symmetric handshake to %s:%d start", t.config.peerIP, t.config.peerConeNatPort)
 	gotIt := false
 	gotMtx := sync.Mutex{}
 	for i := 0; i < SymmetricHandshakeNum; i++ {
@@ -132,7 +135,7 @@ func handshakeS2C(t *P2PTunnel) error {
 		go func(t *P2PTunnel) error {
 			conn, err := net.ListenUDP("udp", nil)
 			if err != nil {
-				gLog.Printf(LevelDEBUG, "listen error")
+				gLog.Printf(LvDEBUG, "listen error")
 				return err
 			}
 			defer conn.Close()
@@ -150,15 +153,15 @@ func handshakeS2C(t *P2PTunnel) error {
 			gotIt = true
 			t.la, _ = net.ResolveUDPAddr("udp", conn.LocalAddr().String())
 			if head.MainType == MsgP2P && head.SubType == MsgPunchHandshake {
-				gLog.Printf(LevelDEBUG, "handshakeS2C read %d handshake ", t.id)
+				gLog.Printf(LvDEBUG, "handshakeS2C read %d handshake ", t.id)
 				UDPWrite(conn, t.ra, MsgP2P, MsgPunchHandshakeAck, P2PHandshakeReq{ID: t.id})
 				_, head, _, _, err = UDPRead(conn, 5000)
 				if err != nil {
-					gLog.Println(LevelDEBUG, "handshakeS2C handshake error")
+					gLog.Println(LvDEBUG, "handshakeS2C handshake error")
 					return err
 				}
 				if head.MainType == MsgP2P && head.SubType == MsgPunchHandshakeAck {
-					gLog.Printf(LevelDEBUG, "handshakeS2C read %d handshake ack %s", t.id, conn.LocalAddr().String())
+					gLog.Printf(LvDEBUG, "handshakeS2C read %d handshake ack %s", t.id, conn.LocalAddr().String())
 					gotCh <- t.la
 					return nil
 				}
@@ -166,15 +169,16 @@ func handshakeS2C(t *P2PTunnel) error {
 			return nil
 		}(t)
 	}
-	gLog.Printf(LevelDEBUG, "send symmetric handshake end")
-	gLog.Println(LevelDEBUG, "handshakeS2C ready, notify peer connect")
+	gLog.Printf(LvDEBUG, "send symmetric handshake end")
+	gLog.Println(LvDEBUG, "handshakeS2C ready, notify peer connect")
 	t.pn.push(t.config.PeerNode, MsgPushHandshakeStart, TunnelMsg{ID: t.id})
 
 	select {
 	case <-time.After(SymmetricHandshakeAckTimeout):
 		return fmt.Errorf("wait handshake failed")
 	case la := <-gotCh:
-		gLog.Println(LevelDEBUG, "symmetric handshake ok", la)
+		gLog.Println(LvDEBUG, "symmetric handshake ok", la)
+		gLog.Printf(LvINFO, "handshakeS2C ok")
 	}
 	return nil
 }

install.go

@@ -0,0 +1,127 @@
+package main
+
+import (
+	"fmt"
+	"io"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"strings"
+	"time"
+)
+
+// examples:
+// listen:
+// ./openp2p install -node hhd1207-222 -token YOUR-TOKEN -sharebandwidth 0
+// listen and build p2papp:
+// ./openp2p install -node hhd1207-222 -token YOUR-TOKEN -sharebandwidth 0 -peernode hhdhome-n1 -dstip 127.0.0.1 -dstport 50022 -protocol tcp -srcport 22
+func install() {
+	gLog.Println(LvINFO, "openp2p start. version: ", OpenP2PVersion)
+	gLog.Println(LvINFO, "Contact: QQ: 477503927, Email: openp2p.cn@gmail.com")
+	gLog.Println(LvINFO, "install start")
+	defer gLog.Println(LvINFO, "install end")
+	// auto uninstall
+	err := os.MkdirAll(defaultInstallPath, 0775)
+
+	if err != nil {
+		gLog.Printf(LvERROR, "MkdirAll %s error:%s", defaultInstallPath, err)
+		return
+	}
+	err = os.Chdir(defaultInstallPath)
+	if err != nil {
+		gLog.Println(LvERROR, "cd error:", err)
+		return
+	}
+
+	uninstall()
+	// save config file
+	parseParams("install")
+	targetPath := filepath.Join(defaultInstallPath, defaultBinName)
+	d := daemon{}
+	// copy files
+
+	binPath, _ := os.Executable()
+	src, errFiles := os.Open(binPath) // can not use args[0], on Windows call openp2p is ok(=openp2p.exe)
+	if errFiles != nil {
+		gLog.Printf(LvERROR, "os.OpenFile %s error:%s", os.Args[0], errFiles)
+		return
+	}
+
+	dst, errFiles := os.OpenFile(targetPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0775)
+	if errFiles != nil {
+		gLog.Printf(LvERROR, "os.OpenFile %s error:%s", targetPath, errFiles)
+		return
+	}
+
+	_, errFiles = io.Copy(dst, src)
+	if errFiles != nil {
+		gLog.Printf(LvERROR, "io.Copy error:%s", errFiles)
+		return
+	}
+	src.Close()
+	dst.Close()
+
+	// install system service
+	gLog.Println(LvINFO, "targetPath:", targetPath)
+	err = d.Control("install", targetPath, []string{"-d"})
+	if err == nil {
+		gLog.Println(LvINFO, "install system service ok.")
+	}
+	time.Sleep(time.Second * 2)
+	err = d.Control("start", targetPath, []string{"-d"})
+	if err != nil {
+		gLog.Println(LvERROR, "start openp2p service error:", err)
+	} else {
+		gLog.Println(LvINFO, "start openp2p service ok.")
+	}
+}
+
+func installByFilename() {
+	params := strings.Split(filepath.Base(os.Args[0]), "-")
+	if len(params) < 4 {
+		return
+	}
+	serverHost := params[1]
+	token := params[2]
+	gLog.Println(LvINFO, "install start")
+	targetPath := os.Args[0]
+	args := []string{"install"}
+	args = append(args, "-serverhost")
+	args = append(args, serverHost)
+	args = append(args, "-token")
+	args = append(args, token)
+	env := os.Environ()
+	cmd := exec.Command(targetPath, args...)
+	cmd.Stdout = os.Stdout
+	cmd.Stderr = os.Stderr
+	cmd.Stdin = os.Stdin
+	cmd.Env = env
+	err := cmd.Run()
+	if err != nil {
+		gLog.Println(LvERROR, "install by filename, start process error:", err)
+		return
+	}
+	gLog.Println(LvINFO, "install end")
+	fmt.Println("Press the Any Key to exit")
+	fmt.Scanln()
+	os.Exit(0)
+}
+func uninstall() {
+	gLog.Println(LvINFO, "uninstall start")
+	defer gLog.Println(LvINFO, "uninstall end")
+	d := daemon{}
+	err := d.Control("stop", "", nil)
+	if err != nil { // service maybe not install
+		return
+	}
+	err = d.Control("uninstall", "", nil)
+	if err != nil {
+		gLog.Println(LvERROR, "uninstall system service error:", err)
+	} else {
+		gLog.Println(LvINFO, "uninstall system service ok.")
+	}
+	binPath := filepath.Join(defaultInstallPath, defaultBinName)
+	os.Remove(binPath + "0")
+	os.Remove(binPath)
+	// os.RemoveAll(defaultInstallPath)  // reserve config.json
+}

log.go

@@ -8,17 +8,15 @@ import (
 	"time"
 )
 
-// LogLevel ...
 type LogLevel int
 
-var gLog *V8log
+var gLog *logger
 
-// LevelDEBUG ...
 const (
-	LevelDEBUG LogLevel = iota
-	LevelINFO
-	LevelWARN
-	LevelERROR
+	LvDEBUG LogLevel = iota
+	LvINFO
+	LvWARN
+	LvERROR
 )
 
 var (
@@ -30,10 +28,10 @@ func init() {
 	logFileNames = make(map[LogLevel]string)
 	loglevel = make(map[LogLevel]string)
 	logFileNames[0] = ".log"
-	loglevel[LevelDEBUG] = "DEBUG"
-	loglevel[LevelINFO] = "INFO"
-	loglevel[LevelWARN] = "WARN"
-	loglevel[LevelERROR] = "ERROR"
+	loglevel[LvDEBUG] = "DEBUG"
+	loglevel[LvINFO] = "INFO"
+	loglevel[LvWARN] = "WARN"
+	loglevel[LvERROR] = "ERROR"
 
 }
 
@@ -43,25 +41,21 @@ const (
 	LogFileAndConsole
 )
 
-// V8log ...
-type V8log struct {
+type logger struct {
 	loggers    map[LogLevel]*log.Logger
 	files      map[LogLevel]*os.File
 	level      LogLevel
-	stopSig    chan bool
 	logDir     string
 	mtx        *sync.Mutex
-	stoped     bool
 	lineEnding string
 	pid        int
 	maxLogSize int64
 	mode       int
 }
 
-// InitLogger ...
-func InitLogger(path string, filePrefix string, level LogLevel, maxLogSize int64, mode int) *V8log {
-	logger := make(map[LogLevel]*log.Logger)
-	openedfile := make(map[LogLevel]*os.File)
+func NewLogger(path string, filePrefix string, level LogLevel, maxLogSize int64, mode int) *logger {
+	loggers := make(map[LogLevel]*log.Logger)
+	logfiles := make(map[LogLevel]*os.File)
 	var (
 		logdir string
 	)
@@ -71,15 +65,15 @@ func InitLogger(path string, filePrefix string, level LogLevel, maxLogSize int64
 		logdir = path + "/log/"
 	}
 	os.MkdirAll(logdir, 0777)
-	for l := range logFileNames {
-		logFilePath := logdir + filePrefix + logFileNames[l]
+	for lv := range logFileNames {
+		logFilePath := logdir + filePrefix + logFileNames[lv]
 		f, err := os.OpenFile(logFilePath, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666)
 		if err != nil {
 			log.Fatal(err)
 		}
 		os.Chmod(logFilePath, 0666)
-		openedfile[l] = f
-		logger[l] = log.New(f, "", log.LstdFlags)
+		logfiles[lv] = f
+		loggers[lv] = log.New(f, "", log.LstdFlags)
 	}
 	var le string
 	if runtime.GOOS == "windows" {
@@ -87,92 +81,84 @@ func InitLogger(path string, filePrefix string, level LogLevel, maxLogSize int64
 	} else {
 		le = "\n"
 	}
-	pLog := &V8log{logger, openedfile, level, make(chan bool, 10), logdir, &sync.Mutex{}, false, le, os.Getpid(), maxLogSize, mode}
+	pLog := &logger{loggers, logfiles, level, logdir, &sync.Mutex{}, le, os.Getpid(), maxLogSize, mode}
 	go pLog.checkFile()
 	return pLog
 }
 
-func (vl *V8log) setLevel(level LogLevel) {
-	vl.mtx.Lock()
-	defer vl.mtx.Unlock()
-	vl.level = level
+func (l *logger) setLevel(level LogLevel) {
+	l.mtx.Lock()
+	defer l.mtx.Unlock()
+	l.level = level
+}
+func (l *logger) setMode(mode int) {
+	l.mtx.Lock()
+	defer l.mtx.Unlock()
+	l.mode = mode
 }
 
-func (vl *V8log) checkFile() {
-	if vl.maxLogSize <= 0 {
+func (l *logger) checkFile() {
+	if l.maxLogSize <= 0 {
 		return
 	}
 	ticker := time.NewTicker(time.Minute)
 	for {
 		select {
 		case <-ticker.C:
-			vl.mtx.Lock()
-			for l, logFile := range vl.files {
+			l.mtx.Lock()
+			for lv, logFile := range l.files {
 				f, e := logFile.Stat()
 				if e != nil {
-					break
+					continue
 				}
-				if f.Size() <= vl.maxLogSize {
-					break
+				if f.Size() <= l.maxLogSize {
+					continue
 				}
 				logFile.Close()
 				fname := f.Name()
-				backupPath := vl.logDir + fname + ".0"
+				backupPath := l.logDir + fname + ".0"
 				os.Remove(backupPath)
-				os.Rename(vl.logDir+fname, backupPath)
-				newFile, e := os.OpenFile(vl.logDir+fname, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666)
+				os.Rename(l.logDir+fname, backupPath)
+				newFile, e := os.OpenFile(l.logDir+fname, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666)
 				if e == nil {
-					vl.loggers[l].SetOutput(newFile)
-					vl.files[l] = newFile
+					l.loggers[lv].SetOutput(newFile)
+					l.files[lv] = newFile
 				}
-
 			}
-			vl.mtx.Unlock()
-		case <-vl.stopSig:
-		}
-		if vl.stoped {
-			break
+			l.mtx.Unlock()
 		}
 	}
 }
 
-// Printf  Warning: report error log depends on this Print format.
-func (vl *V8log) Printf(level LogLevel, format string, params ...interface{}) {
-	vl.mtx.Lock()
-	defer vl.mtx.Unlock()
-	if vl.stoped {
+func (l *logger) Printf(level LogLevel, format string, params ...interface{}) {
+	l.mtx.Lock()
+	defer l.mtx.Unlock()
+	if level < l.level {
 		return
 	}
-	if level < vl.level {
-		return
-	}
-	pidAndLevel := []interface{}{vl.pid, loglevel[level]}
+	pidAndLevel := []interface{}{l.pid, loglevel[level]}
 	params = append(pidAndLevel, params...)
-	if vl.mode == LogFile || vl.mode == LogFileAndConsole {
-		vl.loggers[0].Printf("%d %s "+format+vl.lineEnding, params...)
+	if l.mode == LogFile || l.mode == LogFileAndConsole {
+		l.loggers[0].Printf("%d %s "+format+l.lineEnding, params...)
 	}
-	if vl.mode == LogConsole || vl.mode == LogFileAndConsole {
-		log.Printf("%d %s "+format+vl.lineEnding, params...)
+	if l.mode == LogConsole || l.mode == LogFileAndConsole {
+		log.Printf("%d %s "+format+l.lineEnding, params...)
 	}
 }
 
-// Println ...
-func (vl *V8log) Println(level LogLevel, params ...interface{}) {
-	vl.mtx.Lock()
-	defer vl.mtx.Unlock()
-	if vl.stoped {
+func (l *logger) Println(level LogLevel, params ...interface{}) {
+	l.mtx.Lock()
+	defer l.mtx.Unlock()
+	if level < l.level {
 		return
 	}
-	if level < vl.level {
-		return
-	}
-	pidAndLevel := []interface{}{vl.pid, " ", loglevel[level], " "}
+	pidAndLevel := []interface{}{l.pid, " ", loglevel[level], " "}
 	params = append(pidAndLevel, params...)
-	params = append(params, vl.lineEnding)
-	if vl.mode == LogFile || vl.mode == LogFileAndConsole {
-		vl.loggers[0].Print(params...)
+	params = append(params, l.lineEnding)
+	if l.mode == LogFile || l.mode == LogFileAndConsole {
+		l.loggers[0].Print(params...)
 	}
-	if vl.mode == LogConsole || vl.mode == LogFileAndConsole {
+	if l.mode == LogConsole || l.mode == LogFileAndConsole {
 		log.Print(params...)
 	}
 }

nat.go

@@ -3,47 +3,114 @@ package main
 import (
 	"encoding/json"
 	"fmt"
+	"log"
 	"math/rand"
 	"net"
+	"strconv"
+	"strings"
 	"time"
+
+	reuse "github.com/openp2p-cn/go-reuseport"
 )
 
-func natTest(serverHost string, serverPort int, localPort int, echoPort int) (publicIP string, isPublicIP int, publicPort int, err error) {
+var echoConn *net.UDPConn
+
+func natTCP(serverHost string, serverPort int, localPort int) (publicIP string, publicPort int) {
+	// dialer := &net.Dialer{
+	// 	LocalAddr: &net.TCPAddr{
+	// 		IP:   net.ParseIP("0.0.0.0"),
+	// 		Port: localPort,
+	// 	},
+	// }
+	conn, err := reuse.DialTimeout("tcp4", fmt.Sprintf("%s:%d", "0.0.0.0", localPort), fmt.Sprintf("%s:%d", serverHost, serverPort), time.Second*5)
+	// conn, err := net.Dial("tcp4", fmt.Sprintf("%s:%d", serverHost, serverPort))
+	if err != nil {
+		fmt.Printf("Dial tcp4 %s:%d error:%s", serverHost, serverPort, err)
+		return
+	}
+	defer conn.Close()
+	_, wrerr := conn.Write([]byte("1"))
+	if wrerr != nil {
+		fmt.Printf("Write error: %s\n", wrerr)
+		return
+	}
+	b := make([]byte, 1000)
+	conn.SetReadDeadline(time.Now().Add(time.Second * 5))
+	n, rderr := conn.Read(b)
+	if rderr != nil {
+		fmt.Printf("Read error: %s\n", rderr)
+		return
+	}
+	arr := strings.Split(string(b[:n]), ":")
+	if len(arr) < 2 {
+		return
+	}
+	publicIP = arr[0]
+	port, _ := strconv.ParseInt(arr[1], 10, 32)
+	publicPort = int(port)
+	return
+
+}
+func natTest(serverHost string, serverPort int, localPort int, echoPort int) (publicIP string, hasPublicIP int, hasUPNPorNATPMP int, publicPort int, err error) {
 	conn, err := net.ListenPacket("udp", fmt.Sprintf(":%d", localPort))
 	if err != nil {
-		return "", 0, 0, err
+		gLog.Println(LvERROR, "natTest listen udp error:", err)
+		return "", 0, 0, 0, err
 	}
 	defer conn.Close()
 
 	dst, err := net.ResolveUDPAddr("udp", fmt.Sprintf("%s:%d", serverHost, serverPort))
 	if err != nil {
-		return "", 0, 0, err
+		return "", 0, 0, 0, err
 	}
 
 	// The connection can write data to the desired address.
 	msg, err := newMessage(MsgNATDetect, 0, &NatDetectReq{SrcPort: localPort, EchoPort: echoPort})
 	_, err = conn.WriteTo(msg, dst)
 	if err != nil {
-		return "", 0, 0, err
+		return "", 0, 0, 0, err
 	}
 	deadline := time.Now().Add(NatTestTimeout)
 	err = conn.SetReadDeadline(deadline)
 	if err != nil {
-		return "", 0, 0, err
+		return "", 0, 0, 0, err
 	}
 	buffer := make([]byte, 1024)
 	nRead, _, err := conn.ReadFrom(buffer)
 	if err != nil {
-		gLog.Println(LevelERROR, "NAT detect error:", err)
-		return "", 0, 0, err
+		gLog.Println(LvERROR, "NAT detect error:", err)
+		return "", 0, 0, 0, err
 	}
 	natRsp := NatDetectRsp{}
 	err = json.Unmarshal(buffer[openP2PHeaderSize:nRead], &natRsp)
-
+	hasPublicIP = 0
+	hasUPNPorNATPMP = 0
 	// testing for public ip
 	if echoPort != 0 {
-		for {
-			gLog.Printf(LevelDEBUG, "public ip test start %s:%d", natRsp.IP, echoPort)
+		for i := 0; i < 2; i++ {
+			if i == 1 {
+				// test upnp or nat-pmp
+				nat, err := Discover()
+				if err != nil || nat == nil {
+					gLog.Println(LvDEBUG, "could not perform UPNP discover:", err)
+					break
+				}
+				ext, err := nat.GetExternalAddress()
+				if err != nil {
+					gLog.Println(LvDEBUG, "could not perform UPNP external address:", err)
+					break
+				}
+				log.Println("PublicIP:", ext)
+
+				externalPort, err := nat.AddPortMapping("udp", echoPort, echoPort, "openp2p", 30)
+				if err != nil {
+					gLog.Println(LvDEBUG, "could not add udp UPNP port mapping", externalPort)
+					break
+				} else {
+					nat.AddPortMapping("tcp", echoPort, echoPort, "openp2p", 604800)
+				}
+			}
+			gLog.Printf(LvDEBUG, "public ip test start %s:%d", natRsp.IP, echoPort)
 			conn, err := net.ListenUDP("udp", nil)
 			if err != nil {
 				break
@@ -60,59 +127,67 @@ func natTest(serverHost string, serverPort int, localPort int, echoPort int) (pu
 			conn.SetReadDeadline(time.Now().Add(PublicIPEchoTimeout))
 			_, _, err = conn.ReadFromUDP(buf)
 			if err == nil {
-				gLog.Println(LevelDEBUG, "public ip:YES")
-				natRsp.IsPublicIP = 1
-			} else {
-				gLog.Println(LevelDEBUG, "public ip:NO")
+				if i == 1 {
+					gLog.Println(LvDEBUG, "UPNP or NAT-PMP:YES")
+					hasUPNPorNATPMP = 1
+				} else {
+					gLog.Println(LvDEBUG, "public ip:YES")
+					hasPublicIP = 1
+				}
+				break
 			}
-			break
 		}
 	}
-	return natRsp.IP, natRsp.IsPublicIP, natRsp.Port, nil
+	return natRsp.IP, hasPublicIP, hasUPNPorNATPMP, natRsp.Port, nil
 }
 
-func getNATType(host string, udp1 int, udp2 int) (publicIP string, NATType int, err error) {
+func getNATType(host string, udp1 int, udp2 int) (publicIP string, NATType int, hasIPvr int, hasUPNPorNATPMP int, err error) {
 	// the random local port may be used by other.
-	localPort := int(rand.Uint32()%10000 + 50000)
-	echoPort := int(rand.Uint32()%10000 + 50000)
+	localPort := int(rand.Uint32()%15000 + 50000)
+	echoPort := P2PNetworkInstance(nil).config.TCPPort
 	go echo(echoPort)
-	ip1, isPublicIP, port1, err := natTest(host, udp1, localPort, echoPort)
-	gLog.Printf(LevelDEBUG, "local port:%d  nat port:%d", localPort, port1)
+	// _, natPort := natTCP(host, 27181, localPort)
+	// gLog.Println(LvINFO, "nattcp:", natPort)
+	// _, natPort = natTCP(host, 27180, localPort)
+	// gLog.Println(LvINFO, "nattcp:", natPort)
+	ip1, hasIPv4, hasUPNPorNATPMP, port1, err := natTest(host, udp1, localPort, echoPort)
+	gLog.Printf(LvDEBUG, "local port:%d  nat port:%d", localPort, port1)
 	if err != nil {
-		return "", 0, err
+		return "", 0, hasIPv4, hasUPNPorNATPMP, err
 	}
-	if isPublicIP == 1 {
-		return ip1, NATNone, nil
+	if echoConn != nil {
+		echoConn.Close()
+		echoConn = nil
 	}
-	ip2, _, port2, err := natTest(host, udp2, localPort, 0) // 2rd nat test not need testing publicip
-	gLog.Printf(LevelDEBUG, "local port:%d  nat port:%d", localPort, port2)
+	// if hasPublicIP == 1 || hasUPNPorNATPMP == 1 {
+	// 	return ip1, NATNone, hasUPNPorNATPMP, nil
+	// }
+	_, _, _, port2, err := natTest(host, udp2, localPort, 0) // 2rd nat test not need testing publicip
+	gLog.Printf(LvDEBUG, "local port:%d  nat port:%d", localPort, port2)
 	if err != nil {
-		return "", 0, err
-	}
-	if ip1 != ip2 {
-		return "", 0, fmt.Errorf("ip have changed, please retry again")
+		return "", 0, hasIPv4, hasUPNPorNATPMP, err
 	}
 	natType := NATSymmetric
 	if port1 == port2 {
 		natType = NATCone
 	}
-	//TODO: NATNone
-	return ip1, natType, nil
+	return ip1, natType, hasIPv4, hasUPNPorNATPMP, nil
 }
 
 func echo(echoPort int) {
-	conn, err := net.ListenUDP("udp", &net.UDPAddr{IP: net.IPv4zero, Port: echoPort})
+	var err error
+	echoConn, err = net.ListenUDP("udp", &net.UDPAddr{IP: net.IPv4zero, Port: echoPort})
 	if err != nil {
-		gLog.Println(LevelERROR, "echo server listen error:", err)
+		gLog.Println(LvERROR, "echo server listen error:", err)
 		return
 	}
 	buf := make([]byte, 1600)
-	defer conn.Close()
+	// close outside for breaking the ReadFromUDP
 	// wait 5s for echo testing
-	conn.SetReadDeadline(time.Now().Add(time.Second * 5))
-	n, addr, err := conn.ReadFromUDP(buf)
+	echoConn.SetReadDeadline(time.Now().Add(time.Second * 30))
+	n, addr, err := echoConn.ReadFromUDP(buf)
 	if err != nil {
 		return
 	}
-	conn.WriteToUDP(buf[0:n], addr)
+	echoConn.WriteToUDP(buf[0:n], addr)
 }

openp2p.go

@@ -12,8 +12,7 @@ func main() {
 	rand.Seed(time.Now().UnixNano())
 	binDir := filepath.Dir(os.Args[0])
 	os.Chdir(binDir) // for system service
-	gLog = InitLogger(binDir, "openp2p", LevelDEBUG, 1024*1024, LogFileAndConsole)
-	gLog.Println(LevelINFO, "openp2p start. version: ", OpenP2PVersion)
+	gLog = NewLogger(binDir, "openp2p", LvDEBUG, 1024*1024, LogFileAndConsole)
 	// TODO: install sub command, deamon process
 	if len(os.Args) > 1 {
 		switch os.Args[1] {
@@ -21,14 +20,14 @@ func main() {
 			fmt.Println(OpenP2PVersion)
 			return
 		case "update":
-			gLog = InitLogger(filepath.Dir(os.Args[0]), "openp2p", LevelDEBUG, 1024*1024, LogFileAndConsole)
+			gLog = NewLogger(filepath.Dir(os.Args[0]), "openp2p", LvDEBUG, 1024*1024, LogFileAndConsole)
 			targetPath := filepath.Join(defaultInstallPath, defaultBinName)
 			d := daemon{}
 			err := d.Control("restart", targetPath, nil)
 			if err != nil {
-				gLog.Println(LevelERROR, "restart service error:", err)
+				gLog.Println(LvERROR, "restart service error:", err)
 			} else {
-				gLog.Println(LevelINFO, "restart service ok.")
+				gLog.Println(LvINFO, "restart service ok.")
 			}
 			return
 		case "install":
@@ -41,16 +40,24 @@ func main() {
 	} else {
 		installByFilename()
 	}
+	parseParams("")
+	gLog.Println(LvINFO, "openp2p start. version: ", OpenP2PVersion)
+	gLog.Println(LvINFO, "Contact: QQ: 477503927, Email: openp2p.cn@gmail.com")
 
-	parseParams()
-	gLog.Println(LevelINFO, &gConf)
+	if gConf.daemonMode {
+		d := daemon{}
+		d.run()
+		return
+	}
+
+	gLog.Println(LvINFO, &gConf)
 	setFirewall()
 	network := P2PNetworkInstance(&gConf.Network)
 	if ok := network.Connect(30000); !ok {
-		gLog.Println(LevelERROR, "P2PNetwork login error")
+		gLog.Println(LvERROR, "P2PNetwork login error")
 		return
 	}
-	gLog.Println(LevelINFO, "waiting for connection...")
+	gLog.Println(LvINFO, "waiting for connection...")
 	forever := make(chan bool)
 	<-forever
 }

overlay.go

@@ -0,0 +1,150 @@
+package main
+
+import (
+	"bytes"
+	"encoding/binary"
+	"errors"
+	"net"
+	"time"
+)
+
+var ErrDeadlineExceeded error = &DeadlineExceededError{}
+
+// DeadlineExceededError is returned for an expired deadline.
+type DeadlineExceededError struct{}
+
+// Implement the net.Error interface.
+// The string is "i/o timeout" because that is what was returned
+// by earlier Go versions. Changing it may break programs that
+// match on error strings.
+func (e *DeadlineExceededError) Error() string   { return "i/o timeout" }
+func (e *DeadlineExceededError) Timeout() bool   { return true }
+func (e *DeadlineExceededError) Temporary() bool { return true }
+
+// implement io.Writer
+type overlayConn struct {
+	tunnel      *P2PTunnel
+	connTCP     net.Conn
+	id          uint64
+	rtid        uint64
+	running     bool
+	isClient    bool
+	appID       uint64
+	appKey      uint64
+	appKeyBytes []byte
+	// for udp
+	connUDP       *net.UDPConn
+	remoteAddr    net.Addr
+	udpRelayData  chan []byte
+	lastReadUDPTs time.Time
+}
+
+func (oConn *overlayConn) run() {
+	gLog.Printf(LvDEBUG, "%d overlayConn run start", oConn.id)
+	defer gLog.Printf(LvDEBUG, "%d overlayConn run end", oConn.id)
+	oConn.running = true
+	oConn.lastReadUDPTs = time.Now()
+	buffer := make([]byte, ReadBuffLen+PaddingSize)
+	readBuf := buffer[:ReadBuffLen]
+	encryptData := make([]byte, ReadBuffLen+PaddingSize) // 16 bytes for padding
+	tunnelHead := new(bytes.Buffer)
+	relayHead := new(bytes.Buffer)
+	binary.Write(relayHead, binary.LittleEndian, oConn.rtid)
+	binary.Write(tunnelHead, binary.LittleEndian, oConn.id)
+	for oConn.running && oConn.tunnel.isRuning() {
+		buff, dataLen, err := oConn.Read(readBuf)
+		if err != nil {
+			if ne, ok := err.(net.Error); ok && ne.Timeout() {
+				continue
+			}
+			// overlay tcp connection normal close, debug log
+			gLog.Printf(LvDEBUG, "overlayConn %d read error:%s,close it", oConn.id, err)
+			break
+		}
+		payload := buff[:dataLen]
+		if oConn.appKey != 0 {
+			payload, _ = encryptBytes(oConn.appKeyBytes, encryptData, buffer[:dataLen], dataLen)
+		}
+		writeBytes := append(tunnelHead.Bytes(), payload...)
+		if oConn.rtid == 0 {
+			oConn.tunnel.conn.WriteBytes(MsgP2P, MsgOverlayData, writeBytes)
+			gLog.Printf(LvDEBUG, "write overlay data to %d:%d bodylen=%d", oConn.rtid, oConn.id, len(writeBytes))
+		} else {
+			// write raley data
+			all := append(relayHead.Bytes(), encodeHeader(MsgP2P, MsgOverlayData, uint32(len(writeBytes)))...)
+			all = append(all, writeBytes...)
+			oConn.tunnel.conn.WriteBytes(MsgP2P, MsgRelayData, all)
+			gLog.Printf(LvDEBUG, "write relay data to %d:%d bodylen=%d", oConn.rtid, oConn.id, len(writeBytes))
+		}
+	}
+	if oConn.connTCP != nil {
+		oConn.connTCP.Close()
+	}
+	if oConn.connUDP != nil {
+		oConn.connUDP.Close()
+	}
+	oConn.tunnel.overlayConns.Delete(oConn.id)
+	// notify peer disconnect
+	if oConn.isClient {
+		req := OverlayDisconnectReq{ID: oConn.id}
+		if oConn.rtid == 0 {
+			oConn.tunnel.conn.WriteMessage(MsgP2P, MsgOverlayDisconnectReq, &req)
+		} else {
+			// write relay data
+			msg, _ := newMessage(MsgP2P, MsgOverlayDisconnectReq, &req)
+			msgWithHead := append(relayHead.Bytes(), msg...)
+			oConn.tunnel.conn.WriteBytes(MsgP2P, MsgRelayData, msgWithHead)
+		}
+	}
+}
+
+func (oConn *overlayConn) Read(reuseBuff []byte) (buff []byte, n int, err error) {
+	if oConn.connUDP != nil {
+		if time.Now().After(oConn.lastReadUDPTs.Add(time.Minute * 5)) {
+			err = errors.New("udp close")
+			return
+		}
+		if oConn.remoteAddr != nil { // as server
+			select {
+			case buff = <-oConn.udpRelayData:
+				n = len(buff)
+				oConn.lastReadUDPTs = time.Now()
+			case <-time.After(time.Second * 10):
+				err = ErrDeadlineExceeded
+			}
+		} else { // as client
+			oConn.connUDP.SetReadDeadline(time.Now().Add(5 * time.Second))
+			n, _, err = oConn.connUDP.ReadFrom(reuseBuff)
+			if err == nil {
+				oConn.lastReadUDPTs = time.Now()
+			}
+			buff = reuseBuff
+		}
+		return
+	}
+	oConn.connTCP.SetReadDeadline(time.Now().Add(time.Second * 5))
+	n, err = oConn.connTCP.Read(reuseBuff)
+	buff = reuseBuff
+	return
+}
+
+// calling by p2pTunnel
+func (oConn *overlayConn) Write(buff []byte) (n int, err error) {
+	// add mutex when multi-thread calling
+	if oConn.connUDP != nil {
+		if oConn.remoteAddr == nil {
+			n, err = oConn.connUDP.Write(buff)
+		} else {
+			n, err = oConn.connUDP.WriteTo(buff, oConn.remoteAddr)
+		}
+		if err != nil {
+			oConn.running = false
+		}
+		return
+	}
+	n, err = oConn.connTCP.Write(buff)
+	if err != nil {
+		oConn.running = false
+	}
+	return
+}

overlaytcp.go

@@ -1,85 +0,0 @@
-package main
-
-import (
-	"bytes"
-	"encoding/binary"
-	"net"
-	"time"
-)
-
-// implement io.Writer
-type overlayTCP struct {
-	tunnel      *P2PTunnel
-	conn        net.Conn
-	id          uint64
-	rtid        uint64
-	running     bool
-	isClient    bool
-	appID       uint64
-	appKey      uint64
-	appKeyBytes []byte
-}
-
-func (otcp *overlayTCP) run() {
-	gLog.Printf(LevelDEBUG, "%d overlayTCP run start", otcp.id)
-	defer gLog.Printf(LevelDEBUG, "%d overlayTCP run end", otcp.id)
-	otcp.running = true
-	buffer := make([]byte, ReadBuffLen+PaddingSize)
-	readBuf := buffer[:ReadBuffLen]
-	encryptData := make([]byte, ReadBuffLen+PaddingSize) // 16 bytes for padding
-	tunnelHead := new(bytes.Buffer)
-	relayHead := new(bytes.Buffer)
-	binary.Write(relayHead, binary.LittleEndian, otcp.rtid)
-	binary.Write(tunnelHead, binary.LittleEndian, otcp.id)
-	for otcp.running && otcp.tunnel.isRuning() {
-		otcp.conn.SetReadDeadline(time.Now().Add(time.Second * 5))
-		dataLen, err := otcp.conn.Read(readBuf)
-		if err != nil {
-			if ne, ok := err.(net.Error); ok && ne.Timeout() {
-				continue
-			}
-			// overlay tcp connection normal close, debug log
-			gLog.Printf(LevelDEBUG, "overlayTCP %d read error:%s,close it", otcp.id, err)
-			break
-		} else {
-			payload := readBuf[:dataLen]
-			if otcp.appKey != 0 {
-				payload, _ = encryptBytes(otcp.appKeyBytes, encryptData, buffer[:dataLen], dataLen)
-			}
-			writeBytes := append(tunnelHead.Bytes(), payload...)
-			if otcp.rtid == 0 {
-				otcp.tunnel.conn.WriteBytes(MsgP2P, MsgOverlayData, writeBytes)
-			} else {
-				// write raley data
-				all := append(relayHead.Bytes(), encodeHeader(MsgP2P, MsgOverlayData, uint32(len(writeBytes)))...)
-				all = append(all, writeBytes...)
-				otcp.tunnel.conn.WriteBytes(MsgP2P, MsgRelayData, all)
-				gLog.Printf(LevelDEBUG, "write relay data to %d:%d bodylen=%d", otcp.rtid, otcp.id, len(writeBytes))
-			}
-		}
-	}
-	otcp.conn.Close()
-	otcp.tunnel.overlayConns.Delete(otcp.id)
-	// notify peer disconnect
-	if otcp.isClient {
-		req := OverlayDisconnectReq{ID: otcp.id}
-		if otcp.rtid == 0 {
-			otcp.tunnel.conn.WriteMessage(MsgP2P, MsgOverlayDisconnectReq, &req)
-		} else {
-			// write relay data
-			msg, _ := newMessage(MsgP2P, MsgOverlayDisconnectReq, &req)
-			msgWithHead := append(relayHead.Bytes(), msg...)
-			otcp.tunnel.conn.WriteBytes(MsgP2P, MsgRelayData, msgWithHead)
-		}
-	}
-}
-
-// calling by p2pTunnel
-func (otcp *overlayTCP) Write(buff []byte) (n int, err error) {
-	// add mutex when multi-thread calling
-	n, err = otcp.conn.Write(buff)
-	if err != nil {
-		otcp.tunnel.overlayConns.Delete(otcp.id)
-	}
-	return
-}

p2papp.go

@@ -6,22 +6,26 @@ import (
 	"fmt"
 	"math/rand"
 	"net"
+	"strconv"
+	"strings"
 	"sync"
 	"time"
 )
 
 type p2pApp struct {
-	config    AppConfig
-	listener  net.Listener
-	tunnel    *P2PTunnel
-	rtid      uint64
-	relayNode string
-	hbTime    time.Time
-	hbMtx     sync.Mutex
-	running   bool
-	id        uint64
-	key       uint64
-	wg        sync.WaitGroup
+	config      AppConfig
+	listener    net.Listener
+	listenerUDP *net.UDPConn
+	tunnel      *P2PTunnel
+	rtid        uint64 // relay tunnelID
+	relayNode   string
+	relayMode   string
+	hbTime      time.Time
+	hbMtx       sync.Mutex
+	running     bool
+	id          uint64
+	key         uint64
+	wg          sync.WaitGroup
 }
 
 func (app *p2pApp) isActive() bool {
@@ -44,38 +48,42 @@ func (app *p2pApp) updateHeartbeat() {
 }
 
 func (app *p2pApp) listenTCP() error {
+	gLog.Printf(LvDEBUG, "tcp accept on port %d start", app.config.SrcPort)
+	defer gLog.Printf(LvDEBUG, "tcp accept on port %d end", app.config.SrcPort)
 	var err error
 	app.listener, err = net.Listen("tcp4", fmt.Sprintf("0.0.0.0:%d", app.config.SrcPort))
 	if err != nil {
-		gLog.Printf(LevelERROR, "listen error:%s", err)
+		gLog.Printf(LvERROR, "listen error:%s", err)
 		return err
 	}
-	for {
+	for app.running {
 		conn, err := app.listener.Accept()
 		if err != nil {
-			gLog.Printf(LevelERROR, "%d accept error:%s", app.tunnel.id, err)
+			if app.running {
+				gLog.Printf(LvERROR, "%d accept error:%s", app.id, err)
+			}
 			break
 		}
-		otcp := overlayTCP{
+		oConn := overlayConn{
 			tunnel:   app.tunnel,
-			conn:     conn,
+			connTCP:  conn,
 			id:       rand.Uint64(),
 			isClient: true,
 			rtid:     app.rtid,
 			appID:    app.id,
 			appKey:   app.key,
 		}
-		// calc key bytes for encrypt
-		if otcp.appKey != 0 {
+		// pre-calc key bytes for encrypt
+		if oConn.appKey != 0 {
 			encryptKey := make([]byte, AESKeySize)
-			binary.LittleEndian.PutUint64(encryptKey, otcp.appKey)
-			binary.LittleEndian.PutUint64(encryptKey[8:], otcp.appKey)
-			otcp.appKeyBytes = encryptKey
+			binary.LittleEndian.PutUint64(encryptKey, oConn.appKey)
+			binary.LittleEndian.PutUint64(encryptKey[8:], oConn.appKey)
+			oConn.appKeyBytes = encryptKey
 		}
-		app.tunnel.overlayConns.Store(otcp.id, &otcp)
-		gLog.Printf(LevelDEBUG, "Accept overlayID:%d", otcp.id)
+		app.tunnel.overlayConns.Store(oConn.id, &oConn)
+		gLog.Printf(LvDEBUG, "Accept TCP overlayID:%d", oConn.id)
 		// tell peer connect
-		req := OverlayConnectReq{ID: otcp.id,
+		req := OverlayConnectReq{ID: oConn.id,
 			Token:    app.tunnel.pn.config.Token,
 			DstIP:    app.config.DstHost,
 			DstPort:  app.config.DstPort,
@@ -92,29 +100,117 @@ func (app *p2pApp) listenTCP() error {
 			msgWithHead := append(relayHead.Bytes(), msg...)
 			app.tunnel.conn.WriteBytes(MsgP2P, MsgRelayData, msgWithHead)
 		}
+		go oConn.run()
+	}
+	return nil
+}
 
-		go otcp.run()
+func (app *p2pApp) listenUDP() error {
+	gLog.Printf(LvDEBUG, "udp accept on port %d start", app.config.SrcPort)
+	defer gLog.Printf(LvDEBUG, "udp accept on port %d end", app.config.SrcPort)
+	var err error
+	app.listenerUDP, err = net.ListenUDP("udp", &net.UDPAddr{IP: net.IPv4zero, Port: app.config.SrcPort})
+	if err != nil {
+		gLog.Printf(LvERROR, "listen error:%s", err)
+		return err
+	}
+	buffer := make([]byte, 64*1024)
+	udpID := make([]byte, 8)
+	for {
+		app.listenerUDP.SetReadDeadline(time.Now().Add(time.Second * 10))
+		len, remoteAddr, err := app.listenerUDP.ReadFrom(buffer)
+		if err != nil {
+			if ne, ok := err.(net.Error); ok && ne.Timeout() {
+				continue
+			} else {
+				gLog.Printf(LvERROR, "udp read failed:%s", err)
+				break
+			}
+		} else {
+			b := bytes.Buffer{}
+			b.Write(buffer[:len])
+			// load from app.tunnel.overlayConns by remoteAddr error, new udp connection
+			remoteIP := strings.Split(remoteAddr.String(), ":")[0]
+			port, _ := strconv.Atoi(strings.Split(remoteAddr.String(), ":")[1])
+			a := net.ParseIP(remoteIP)
+			udpID[0] = a[0]
+			udpID[1] = a[1]
+			udpID[2] = a[2]
+			udpID[3] = a[3]
+			udpID[4] = byte(port)
+			udpID[5] = byte(port >> 8)
+			id := binary.LittleEndian.Uint64(udpID)
+			s, ok := app.tunnel.overlayConns.Load(id)
+			if !ok {
+				oConn := overlayConn{
+					tunnel:       app.tunnel,
+					connUDP:      app.listenerUDP,
+					remoteAddr:   remoteAddr,
+					udpRelayData: make(chan []byte, 1000),
+					id:           id,
+					isClient:     true,
+					rtid:         app.rtid,
+					appID:        app.id,
+					appKey:       app.key,
+				}
+				// calc key bytes for encrypt
+				if oConn.appKey != 0 {
+					encryptKey := make([]byte, AESKeySize)
+					binary.LittleEndian.PutUint64(encryptKey, oConn.appKey)
+					binary.LittleEndian.PutUint64(encryptKey[8:], oConn.appKey)
+					oConn.appKeyBytes = encryptKey
+				}
+				app.tunnel.overlayConns.Store(oConn.id, &oConn)
+				gLog.Printf(LvDEBUG, "Accept UDP overlayID:%d", oConn.id)
+				// tell peer connect
+				req := OverlayConnectReq{ID: oConn.id,
+					Token:    app.tunnel.pn.config.Token,
+					DstIP:    app.config.DstHost,
+					DstPort:  app.config.DstPort,
+					Protocol: app.config.Protocol,
+					AppID:    app.id,
+				}
+				if app.rtid == 0 {
+					app.tunnel.conn.WriteMessage(MsgP2P, MsgOverlayConnectReq, &req)
+				} else {
+					req.RelayTunnelID = app.tunnel.id
+					relayHead := new(bytes.Buffer)
+					binary.Write(relayHead, binary.LittleEndian, app.rtid)
+					msg, _ := newMessage(MsgP2P, MsgOverlayConnectReq, &req)
+					msgWithHead := append(relayHead.Bytes(), msg...)
+					app.tunnel.conn.WriteBytes(MsgP2P, MsgRelayData, msgWithHead)
+				}
+				go oConn.run()
+				oConn.udpRelayData <- b.Bytes()
+			}
+
+			// load from app.tunnel.overlayConns by remoteAddr ok, write relay data
+			overlayConn, ok := s.(*overlayConn)
+			if !ok {
+				continue
+			}
+			overlayConn.udpRelayData <- b.Bytes()
+		}
 	}
 	return nil
 }
 
 func (app *p2pApp) listen() error {
-	gLog.Printf(LevelINFO, "LISTEN ON PORT %d START", app.config.SrcPort)
-	defer gLog.Printf(LevelINFO, "LISTEN ON PORT %d START", app.config.SrcPort)
+	gLog.Printf(LvINFO, "LISTEN ON PORT %s:%d START", app.config.Protocol, app.config.SrcPort)
+	defer gLog.Printf(LvINFO, "LISTEN ON PORT %s:%d END", app.config.Protocol, app.config.SrcPort)
 	app.wg.Add(1)
 	defer app.wg.Done()
 	app.running = true
 	if app.rtid != 0 {
 		go app.relayHeartbeatLoop()
 	}
-	for app.running {
+	for app.tunnel.isRuning() && app.running {
 		if app.config.Protocol == "udp" {
-			app.listenTCP()
+			app.listenUDP()
 		} else {
 			app.listenTCP()
 		}
-		time.Sleep(time.Second * 5)
-		// TODO: listen UDP
+		time.Sleep(time.Second * 10)
 	}
 	return nil
 }
@@ -124,6 +220,9 @@ func (app *p2pApp) close() {
 	if app.listener != nil {
 		app.listener.Close()
 	}
+	if app.listenerUDP != nil {
+		app.listenerUDP.Close()
+	}
 	if app.tunnel != nil {
 		app.tunnel.closeOverlayConns(app.id)
 	}
@@ -134,8 +233,8 @@ func (app *p2pApp) close() {
 func (app *p2pApp) relayHeartbeatLoop() {
 	app.wg.Add(1)
 	defer app.wg.Done()
-	gLog.Printf(LevelDEBUG, "relayHeartbeat to %d start", app.rtid)
-	defer gLog.Printf(LevelDEBUG, "relayHeartbeat to %d end", app.rtid)
+	gLog.Printf(LvDEBUG, "relayHeartbeat to %d start", app.rtid)
+	defer gLog.Printf(LvDEBUG, "relayHeartbeat to %d end", app.rtid)
 	relayHead := new(bytes.Buffer)
 	binary.Write(relayHead, binary.LittleEndian, app.rtid)
 	req := RelayHeartbeat{RelayTunnelID: app.tunnel.id,

p2pnetwork.go

@@ -7,8 +7,9 @@ import (
 	"encoding/json"
 	"errors"
 	"fmt"
+	"math"
 	"math/rand"
-	"net"
+	"net/http"
 	"net/url"
 	"strings"
 	"sync"
@@ -75,7 +76,7 @@ func (pn *P2PNetwork) run() {
 			time.Sleep(NetworkHeartbeatTime)
 			err := pn.init()
 			if err != nil {
-				gLog.Println(LevelERROR, "P2PNetwork init error:", err)
+				gLog.Println(LvERROR, "P2PNetwork init error:", err)
 			}
 		}
 	}
@@ -93,9 +94,14 @@ func (pn *P2PNetwork) Connect(timeout int) bool {
 }
 
 func (pn *P2PNetwork) runAll() {
-	gConf.mtx.Lock()
+	gConf.mtx.Lock() // lock for copy gConf.Apps and the modification of config(it's pointer)
 	defer gConf.mtx.Unlock()
-	for _, config := range gConf.Apps {
+	allApps := gConf.Apps // read a copy, other thread will modify the gConf.Apps
+
+	for _, config := range allApps {
+		if config.nextRetryTime.After(time.Now()) {
+			continue
+		}
 		if config.Enabled == 0 {
 			continue
 		}
@@ -103,37 +109,44 @@ func (pn *P2PNetwork) runAll() {
 			config.AppName = fmt.Sprintf("%s%d", config.Protocol, config.SrcPort)
 		}
 		appExist := false
-		appActive := false
+		var appID uint64
 		i, ok := pn.apps.Load(fmt.Sprintf("%s%d", config.Protocol, config.SrcPort))
 		if ok {
 			app := i.(*p2pApp)
 			appExist = true
+			appID = app.id
 			if app.isActive() {
-				appActive = true
-			}
-		}
-		if appExist && appActive {
-			continue
-		}
-		if appExist && !appActive {
-			gLog.Printf(LevelINFO, "detect app %s disconnect, reconnecting...", config.AppName)
-			pn.DeleteApp(config)
-			if config.retryTime.Add(time.Minute * 15).Before(time.Now()) {
-				config.retryNum = 0
-			}
-			config.retryNum++
-			config.retryTime = time.Now()
-			if config.retryNum > MaxRetry {
-				gLog.Printf(LevelERROR, "app %s%d retry more than %d times, exit.", config.Protocol, config.SrcPort, MaxRetry)
 				continue
 			}
 		}
-		go pn.AddApp(config)
+		if appExist {
+			pn.DeleteApp(*config)
+		}
+		if config.retryNum > 0 {
+			gLog.Printf(LvINFO, "detect app %s(%d) disconnect, reconnecting the %d times...", config.AppName, appID, config.retryNum)
+			if time.Now().Add(-time.Minute * 15).After(config.retryTime) { // normal lasts 15min
+				config.retryNum = 0
+			}
+		}
+		config.retryNum++
+		config.retryTime = time.Now()
+		increase := math.Pow(1.3, float64(config.retryNum))
+		if increase > 900 {
+			increase = 900
+		}
+		config.nextRetryTime = time.Now().Add(time.Second * time.Duration(increase)) // exponential increase retry time. 1.3^x
+		config.connectTime = time.Now()
+		config.peerToken = pn.config.Token
+		gConf.mtx.Unlock() // AddApp will take a period of time
+		err := pn.AddApp(*config)
+		gConf.mtx.Lock()
+		if err != nil {
+			config.errMsg = err.Error()
+		}
 	}
 }
 func (pn *P2PNetwork) autorunApp() {
-	gLog.Println(LevelINFO, "autorunApp start")
-	// TODO: use gConf to check reconnect
+	gLog.Println(LvINFO, "autorunApp start")
 	for pn.running {
 		time.Sleep(time.Second)
 		if !pn.online {
@@ -142,65 +155,66 @@ func (pn *P2PNetwork) autorunApp() {
 		pn.runAll()
 		time.Sleep(time.Second * 10)
 	}
-	gLog.Println(LevelINFO, "autorunApp end")
+	gLog.Println(LvINFO, "autorunApp end")
 }
 
-func (pn *P2PNetwork) addRelayTunnel(config AppConfig, appid uint64, appkey uint64) (*P2PTunnel, uint64, error) {
-	gLog.Printf(LevelINFO, "addRelayTunnel to %s start", config.PeerNode)
-	defer gLog.Printf(LevelINFO, "addRelayTunnel to %s end", config.PeerNode)
+func (pn *P2PNetwork) addRelayTunnel(config AppConfig) (*P2PTunnel, uint64, string, error) {
+	gLog.Printf(LvINFO, "addRelayTunnel to %s start", config.PeerNode)
+	defer gLog.Printf(LvINFO, "addRelayTunnel to %s end", config.PeerNode)
+	// request a relay node or specify manually(TODO)
 	pn.write(MsgRelay, MsgRelayNodeReq, &RelayNodeReq{config.PeerNode})
 	head, body := pn.read("", MsgRelay, MsgRelayNodeRsp, time.Second*10)
 	if head == nil {
-		return nil, 0, errors.New("read MsgRelayNodeRsp error")
+		return nil, 0, "", errors.New("read MsgRelayNodeRsp error")
 	}
 	rsp := RelayNodeRsp{}
 	err := json.Unmarshal(body, &rsp)
 	if err != nil {
-		gLog.Printf(LevelERROR, "wrong RelayNodeRsp:%s", err)
-		return nil, 0, errors.New("unmarshal MsgRelayNodeRsp error")
+		gLog.Printf(LvERROR, "wrong RelayNodeRsp:%s", err)
+		return nil, 0, "", errors.New("unmarshal MsgRelayNodeRsp error")
 	}
 	if rsp.RelayName == "" || rsp.RelayToken == 0 {
-		gLog.Printf(LevelERROR, "MsgRelayNodeReq error")
-		return nil, 0, errors.New("MsgRelayNodeReq error")
+		gLog.Printf(LvERROR, "MsgRelayNodeReq error")
+		return nil, 0, "", errors.New("MsgRelayNodeReq error")
 	}
-	gLog.Printf(LevelINFO, "got relay node:%s", rsp.RelayName)
+	gLog.Printf(LvINFO, "got relay node:%s", rsp.RelayName)
 	relayConfig := config
 	relayConfig.PeerNode = rsp.RelayName
 	relayConfig.peerToken = rsp.RelayToken
+	///
 	t, err := pn.addDirectTunnel(relayConfig, 0)
 	if err != nil {
-		gLog.Println(LevelERROR, "direct connect error:", err)
-		return nil, 0, err
+		gLog.Println(LvERROR, "direct connect error:", err)
+		return nil, 0, "", err
 	}
 	// notify peer addRelayTunnel
 	req := AddRelayTunnelReq{
 		From:       pn.config.Node,
 		RelayName:  rsp.RelayName,
 		RelayToken: rsp.RelayToken,
-		AppID:      appid,
-		AppKey:     appkey,
 	}
-	gLog.Printf(LevelINFO, "push relay %s---------%s", config.PeerNode, rsp.RelayName)
+	gLog.Printf(LvINFO, "push relay %s---------%s", config.PeerNode, rsp.RelayName)
 	pn.push(config.PeerNode, MsgPushAddRelayTunnelReq, &req)
 
 	// wait relay ready
 	head, body = pn.read(config.PeerNode, MsgPush, MsgPushAddRelayTunnelRsp, PeerAddRelayTimeount) // TODO: const value
 	if head == nil {
-		gLog.Printf(LevelERROR, "read MsgPushAddRelayTunnelRsp error")
-		return nil, 0, errors.New("read MsgPushAddRelayTunnelRsp error")
+		gLog.Printf(LvERROR, "read MsgPushAddRelayTunnelRsp error")
+		return nil, 0, "", errors.New("read MsgPushAddRelayTunnelRsp error")
 	}
 	rspID := TunnelMsg{}
 	err = json.Unmarshal(body, &rspID)
 	if err != nil {
-		gLog.Printf(LevelERROR, "wrong RelayNodeRsp:%s", err)
-		return nil, 0, errors.New("unmarshal MsgRelayNodeRsp error")
+		gLog.Printf(LvERROR, "wrong RelayNodeRsp:%s", err)
+		return nil, 0, "", errors.New("unmarshal MsgRelayNodeRsp error")
 	}
-	return t, rspID.ID, err
+	return t, rspID.ID, rsp.Mode, err
 }
 
+// use *AppConfig to save status
 func (pn *P2PNetwork) AddApp(config AppConfig) error {
-	gLog.Printf(LevelINFO, "addApp %s to %s:%s:%d start", config.AppName, config.PeerNode, config.DstHost, config.DstPort)
-	defer gLog.Printf(LevelINFO, "addApp %s to %s:%s:%d end", config.AppName, config.PeerNode, config.DstHost, config.DstPort)
+	gLog.Printf(LvINFO, "addApp %s to %s:%s:%d start", config.AppName, config.PeerNode, config.DstHost, config.DstPort)
+	defer gLog.Printf(LvINFO, "addApp %s to %s:%s:%d end", config.AppName, config.PeerNode, config.DstHost, config.DstPort)
 	if !pn.online {
 		return errors.New("P2PNetwork offline")
 	}
@@ -215,24 +229,27 @@ func (pn *P2PNetwork) AddApp(config AppConfig) error {
 	}
 	appID := rand.Uint64()
 	appKey := uint64(0)
-	t, err := pn.addDirectTunnel(config, 0)
 	var rtid uint64
 	relayNode := ""
+	relayMode := ""
 	peerNatType := NATUnknown
 	peerIP := ""
 	errMsg := ""
-	if err != nil && err == ErrorHandshake {
-		gLog.Println(LevelERROR, "direct connect failed, try to relay")
-		appKey = rand.Uint64()
-		t, rtid, err = pn.addRelayTunnel(config, appID, appKey)
-		if t != nil {
-			relayNode = t.config.PeerNode
-		}
-	}
+	t, err := pn.addDirectTunnel(config, 0)
 	if t != nil {
 		peerNatType = t.config.peerNatType
 		peerIP = t.config.peerIP
 	}
+	// TODO: if tcp failed, should try udp punching, nattype should refactor also, when NATNONE and failed we don't know the peerNatType
+
+	if err != nil && err == ErrorHandshake {
+		gLog.Println(LvERROR, "direct connect failed, try to relay")
+		t, rtid, relayMode, err = pn.addRelayTunnel(config)
+		if t != nil {
+			relayNode = t.config.PeerNode
+		}
+	}
+
 	if err != nil {
 		errMsg = err.Error()
 	}
@@ -254,6 +271,16 @@ func (pn *P2PNetwork) AddApp(config AppConfig) error {
 	if err != nil {
 		return err
 	}
+	if rtid != 0 || t.conn.Protocol() == "tcp" {
+		// sync appkey
+		appKey = rand.Uint64()
+		req := APPKeySync{
+			AppID:  appID,
+			AppKey: appKey,
+		}
+		gLog.Printf(LvINFO, "sync appkey to %s", config.PeerNode)
+		pn.push(config.PeerNode, MsgPushAPPKey, &req)
+	}
 	app := p2pApp{
 		id:        appID,
 		key:       appKey,
@@ -261,6 +288,7 @@ func (pn *P2PNetwork) AddApp(config AppConfig) error {
 		config:    config,
 		rtid:      rtid,
 		relayNode: relayNode,
+		relayMode: relayMode,
 		hbTime:    time.Now()}
 	pn.apps.Store(fmt.Sprintf("%s%d", config.Protocol, config.SrcPort), &app)
 	if err == nil {
@@ -270,21 +298,21 @@ func (pn *P2PNetwork) AddApp(config AppConfig) error {
 }
 
 func (pn *P2PNetwork) DeleteApp(config AppConfig) {
-	gLog.Printf(LevelINFO, "DeleteApp %s%d start", config.Protocol, config.SrcPort)
-	defer gLog.Printf(LevelINFO, "DeleteApp %s%d end", config.Protocol, config.SrcPort)
+	gLog.Printf(LvINFO, "DeleteApp %s%d start", config.Protocol, config.SrcPort)
+	defer gLog.Printf(LvINFO, "DeleteApp %s%d end", config.Protocol, config.SrcPort)
 	// close the apps of this config
 	i, ok := pn.apps.Load(fmt.Sprintf("%s%d", config.Protocol, config.SrcPort))
 	if ok {
 		app := i.(*p2pApp)
-		gLog.Printf(LevelINFO, "app %s exist, delete it", fmt.Sprintf("%s%d", config.Protocol, config.SrcPort))
+		gLog.Printf(LvINFO, "app %s exist, delete it", fmt.Sprintf("%s%d", config.Protocol, config.SrcPort))
 		app.close()
 		pn.apps.Delete(fmt.Sprintf("%s%d", config.Protocol, config.SrcPort))
 	}
 }
 
 func (pn *P2PNetwork) addDirectTunnel(config AppConfig, tid uint64) (*P2PTunnel, error) {
-	gLog.Printf(LevelDEBUG, "addDirectTunnel %s%d to %s:%s:%d start", config.Protocol, config.SrcPort, config.PeerNode, config.DstHost, config.DstPort)
-	defer gLog.Printf(LevelDEBUG, "addDirectTunnel %s%d to %s:%s:%d end", config.Protocol, config.SrcPort, config.PeerNode, config.DstHost, config.DstPort)
+	gLog.Printf(LvDEBUG, "addDirectTunnel %s%d to %s:%s:%d start", config.Protocol, config.SrcPort, config.PeerNode, config.DstHost, config.DstPort)
+	defer gLog.Printf(LvDEBUG, "addDirectTunnel %s%d to %s:%s:%d end", config.Protocol, config.SrcPort, config.PeerNode, config.DstHost, config.DstPort)
 	isClient := false
 	// client side tid=0, assign random uint64
 	if tid == 0 {
@@ -304,11 +332,11 @@ func (pn *P2PNetwork) addDirectTunnel(config AppConfig, tid uint64) (*P2PTunnel,
 			}
 
 			// client side checking
-			gLog.Println(LevelINFO, "tunnel already exist ", config.PeerNode)
+			gLog.Println(LvINFO, "tunnel already exist ", config.PeerNode)
 			isActive := t.checkActive()
 			// inactive, close it
 			if !isActive {
-				gLog.Println(LevelINFO, "but it's not active, close it ", config.PeerNode)
+				gLog.Println(LvINFO, "but it's not active, close it ", config.PeerNode)
 				t.close()
 			} else {
 				// active
@@ -318,59 +346,119 @@ func (pn *P2PNetwork) addDirectTunnel(config AppConfig, tid uint64) (*P2PTunnel,
 		}
 		return true
 	})
+	if exist {
+		return t, nil
+	}
 	// create tunnel if not exist
-	if !exist {
-		t = &P2PTunnel{pn: pn,
-			config: config,
-			id:     tid,
+	t = &P2PTunnel{pn: pn,
+		config: config,
+		id:     tid,
+	}
+	pn.msgMapMtx.Lock()
+	pn.msgMap[nodeNameToID(config.PeerNode)] = make(chan []byte, 50)
+	pn.msgMapMtx.Unlock()
+	// server side
+	if !isClient {
+		err := pn.newTunnel(t, tid, isClient)
+		return t, err // always return
+	}
+	// client side
+	// peer info
+	initErr := t.requestPeerInfo()
+	if initErr != nil {
+		gLog.Println(LvERROR, "init error:", initErr)
+		return nil, initErr
+	}
+	err := ErrorHandshake
+	// try TCP6
+	if IsIPv6(t.config.peerIPv6) && IsIPv6(t.pn.config.publicIPv6) {
+		gLog.Println(LvINFO, "try TCP6")
+		t.config.linkMode = LinkModeTCP6
+		t.config.isUnderlayServer = 0
+		if err = pn.newTunnel(t, tid, isClient); err == nil {
+			return t, nil
 		}
-		pn.msgMapMtx.Lock()
-		pn.msgMap[nodeNameToID(config.PeerNode)] = make(chan []byte, 50)
-		pn.msgMapMtx.Unlock()
-		t.init()
-		if isClient {
-			if err := t.connect(); err != nil {
-				gLog.Println(LevelERROR, "p2pTunnel connect error:", err)
-				return t, err
-			}
+	}
+
+	// TODO: try UDP6
+
+	// try TCP4
+	if t.config.hasIPv4 == 1 || t.pn.config.hasIPv4 == 1 || t.config.hasUPNPorNATPMP == 1 || t.pn.config.hasUPNPorNATPMP == 1 {
+		gLog.Println(LvINFO, "try TCP4")
+		t.config.linkMode = LinkModeTCP4
+		if t.config.hasIPv4 == 1 || t.config.hasUPNPorNATPMP == 1 {
+			t.config.isUnderlayServer = 0
 		} else {
-			rsp := PushConnectRsp{
-				Error:       0,
-				Detail:      "connect ok",
-				To:          t.config.PeerNode,
-				From:        pn.config.Node,
-				NatType:     pn.config.natType,
-				FromIP:      pn.config.publicIP,
-				ConeNatPort: t.coneNatPort,
-				ID:          t.id}
-			t.pn.push(t.config.PeerNode, MsgPushConnectRsp, rsp)
-			if err := t.listen(); err != nil {
-				gLog.Println(LevelERROR, "p2pTunnel listen error:", err)
-				return t, err
-			}
+			t.config.isUnderlayServer = 1
+		}
+		if err = pn.newTunnel(t, tid, isClient); err == nil {
+			return t, nil
+		}
+	}
+	// TODO: try UDP4
+
+	// try TCPPunch
+	if t.config.peerNatType == NATCone && t.pn.config.natType == NATCone { // TODO: support c2s
+		gLog.Println(LvINFO, "try TCP4 Punch")
+		t.config.linkMode = LinkModeTCPPunch
+		t.config.isUnderlayServer = 0
+		if err = pn.newTunnel(t, tid, isClient); err == nil {
+			return t, nil
+		}
+	}
+	// try UDPPunch
+	if t.config.peerNatType == NATCone || t.pn.config.natType == NATCone {
+		gLog.Println(LvINFO, "try UDP4 Punch")
+		t.config.linkMode = LinkModeUDPPunch
+		t.config.isUnderlayServer = 0
+		if err = pn.newTunnel(t, tid, isClient); err == nil {
+			return t, nil
+		}
+	}
+	return nil, err
+}
+
+func (pn *P2PNetwork) newTunnel(t *P2PTunnel, tid uint64, isClient bool) error {
+	t.initPort()
+	if isClient {
+		if err := t.connect(); err != nil {
+			gLog.Println(LvERROR, "p2pTunnel connect error:", err)
+			return err
+		}
+	} else {
+		if err := t.listen(); err != nil {
+			gLog.Println(LvERROR, "p2pTunnel listen error:", err)
+			return err
 		}
 	}
 	// store it when success
-	gLog.Printf(LevelDEBUG, "store tunnel %d", tid)
+	gLog.Printf(LvDEBUG, "store tunnel %d", tid)
 	pn.allTunnels.Store(tid, t)
-	return t, nil
+	return nil
 }
-
 func (pn *P2PNetwork) init() error {
-	gLog.Println(LevelINFO, "init start")
+	gLog.Println(LvINFO, "init start")
 	var err error
 	for {
 		// detect nat type
-		pn.config.publicIP, pn.config.natType, err = getNATType(pn.config.ServerHost, pn.config.UDPPort1, pn.config.UDPPort2)
-		// TODO rm test s2s
+		pn.config.publicIP, pn.config.natType, pn.config.hasIPv4, pn.config.hasUPNPorNATPMP, err = getNATType(pn.config.ServerHost, pn.config.UDPPort1, pn.config.UDPPort2)
+		// for testcase
 		if strings.Contains(pn.config.Node, "openp2pS2STest") {
 			pn.config.natType = NATSymmetric
+			pn.config.hasIPv4 = 0
+			pn.config.hasUPNPorNATPMP = 0
+
+		}
+		if strings.Contains(pn.config.Node, "openp2pC2CTest") {
+			pn.config.natType = NATCone
+			pn.config.hasIPv4 = 0
+			pn.config.hasUPNPorNATPMP = 0
 		}
 		if err != nil {
-			gLog.Println(LevelDEBUG, "detect NAT type error:", err)
+			gLog.Println(LvDEBUG, "detect NAT type error:", err)
 			break
 		}
-		gLog.Println(LevelDEBUG, "detect NAT type:", pn.config.natType, " publicIP:", pn.config.publicIP)
+		gLog.Println(LvDEBUG, "detect NAT type:", pn.config.natType, " publicIP:", pn.config.publicIP)
 		gatewayURL := fmt.Sprintf("%s:%d", pn.config.ServerHost, pn.config.ServerPort)
 		forwardPath := "/openp2p/v1/login"
 		config := tls.Config{InsecureSkipVerify: true} // let's encrypt root cert "DST Root CA X3" expired at 2021/09/29. many old system(windows server 2008 etc) will not trust our cert
@@ -403,28 +491,30 @@ func (pn *P2PNetwork) init() error {
 		pn.config.os = getOsName()
 
 		req := ReportBasic{
-			Mac:     pn.config.mac,
-			LanIP:   pn.config.localIP,
-			OS:      pn.config.os,
-			Version: OpenP2PVersion,
+			Mac:             pn.config.mac,
+			LanIP:           pn.config.localIP,
+			OS:              pn.config.os,
+			HasIPv4:         pn.config.hasIPv4,
+			HasUPNPorNATPMP: pn.config.hasUPNPorNATPMP,
+			Version:         OpenP2PVersion,
 		}
 		rsp := netInfo()
-		gLog.Println(LevelDEBUG, "netinfo:", rsp)
+		gLog.Println(LvDEBUG, "netinfo:", rsp)
 		if rsp != nil && rsp.Country != "" {
-			if len(rsp.IP) == net.IPv6len {
-				pn.config.ipv6 = rsp.IP.String()
+			if IsIPv6(rsp.IP.String()) {
+				pn.config.publicIPv6 = rsp.IP.String()
 				req.IPv6 = rsp.IP.String()
 			}
 			req.NetInfo = *rsp
 		}
 		pn.write(MsgReport, MsgReportBasic, &req)
-		gLog.Println(LevelDEBUG, "P2PNetwork init ok")
+		gLog.Println(LvDEBUG, "P2PNetwork init ok")
 		break
 	}
 	if err != nil {
 		// init failed, retry
 		pn.restartCh <- true
-		gLog.Println(LevelERROR, "P2PNetwork init error:", err)
+		gLog.Println(LvERROR, "P2PNetwork init error:", err)
 	}
 	return err
 }
@@ -433,7 +523,7 @@ func (pn *P2PNetwork) handleMessage(t int, msg []byte) {
 	head := openP2PHeader{}
 	err := binary.Read(bytes.NewReader(msg[:openP2PHeaderSize]), binary.LittleEndian, &head)
 	if err != nil {
-		gLog.Println(LevelERROR, "handleMessage error:", err)
+		gLog.Println(LvERROR, "handleMessage error:", err)
 		return
 	}
 	switch head.MainType {
@@ -442,26 +532,24 @@ func (pn *P2PNetwork) handleMessage(t int, msg []byte) {
 		rsp := LoginRsp{}
 		err = json.Unmarshal(msg[openP2PHeaderSize:], &rsp)
 		if err != nil {
-			gLog.Printf(LevelERROR, "wrong login response:%s", err)
+			gLog.Printf(LvERROR, "wrong login response:%s", err)
 			return
 		}
 		if rsp.Error != 0 {
-			gLog.Printf(LevelERROR, "login error:%d, detail:%s", rsp.Error, rsp.Detail)
+			gLog.Printf(LvERROR, "login error:%d, detail:%s", rsp.Error, rsp.Detail)
 			pn.running = false
 		} else {
 			pn.serverTs = rsp.Ts
 			pn.config.Token = rsp.Token
 			pn.config.User = rsp.User
-			gConf.mtx.Lock()
-			gConf.Network.Token = rsp.Token
-			gConf.Network.User = rsp.User
-			gConf.mtx.Unlock()
+			gConf.setToken(rsp.Token)
+			gConf.setUser(rsp.User)
 			gConf.save()
 			pn.localTs = time.Now().Unix()
-			gLog.Printf(LevelINFO, "login ok. user=%s,Server ts=%d, local ts=%d", rsp.User, rsp.Ts, pn.localTs)
+			gLog.Printf(LvINFO, "login ok. user=%s,Server ts=%d, local ts=%d", rsp.User, rsp.Ts, pn.localTs)
 		}
 	case MsgHeartbeat:
-		gLog.Printf(LevelDEBUG, "P2PNetwork heartbeat ok")
+		gLog.Printf(LvDEBUG, "P2PNetwork heartbeat ok")
 	case MsgPush:
 		handlePush(pn, head.SubType, msg)
 	default:
@@ -474,21 +562,21 @@ func (pn *P2PNetwork) handleMessage(t int, msg []byte) {
 }
 
 func (pn *P2PNetwork) readLoop() {
-	gLog.Printf(LevelDEBUG, "P2PNetwork readLoop start")
+	gLog.Printf(LvDEBUG, "P2PNetwork readLoop start")
 	pn.wg.Add(1)
 	defer pn.wg.Done()
 	for pn.running {
 		pn.conn.SetReadDeadline(time.Now().Add(NetworkHeartbeatTime + 10*time.Second))
 		t, msg, err := pn.conn.ReadMessage()
 		if err != nil {
-			gLog.Printf(LevelERROR, "P2PNetwork read error:%s", err)
+			gLog.Printf(LvERROR, "P2PNetwork read error:%s", err)
 			pn.conn.Close()
 			pn.restartCh <- true
 			break
 		}
 		pn.handleMessage(t, msg)
 	}
-	gLog.Printf(LevelDEBUG, "P2PNetwork readLoop end")
+	gLog.Printf(LvDEBUG, "P2PNetwork readLoop end")
 }
 
 func (pn *P2PNetwork) write(mainType uint16, subType uint16, packet interface{}) error {
@@ -502,14 +590,14 @@ func (pn *P2PNetwork) write(mainType uint16, subType uint16, packet interface{})
 	pn.writeMtx.Lock()
 	defer pn.writeMtx.Unlock()
 	if err = pn.conn.WriteMessage(websocket.BinaryMessage, msg); err != nil {
-		gLog.Printf(LevelERROR, "write msgType %d,%d error:%s", mainType, subType, err)
+		gLog.Printf(LvERROR, "write msgType %d,%d error:%s", mainType, subType, err)
 		pn.conn.Close()
 	}
 	return err
 }
 
 func (pn *P2PNetwork) relay(to uint64, body []byte) error {
-	gLog.Printf(LevelDEBUG, "relay data to %d", to)
+	gLog.Printf(LvDEBUG, "relay data to %d", to)
 	i, ok := pn.allTunnels.Load(to)
 	if !ok {
 		return nil
@@ -523,7 +611,7 @@ func (pn *P2PNetwork) relay(to uint64, body []byte) error {
 }
 
 func (pn *P2PNetwork) push(to string, subType uint16, packet interface{}) error {
-	gLog.Printf(LevelDEBUG, "push msgType %d to %s", subType, to)
+	gLog.Printf(LvDEBUG, "push msgType %d to %s", subType, to)
 	if !pn.online {
 		return errors.New("client offline")
 	}
@@ -545,7 +633,7 @@ func (pn *P2PNetwork) push(to string, subType uint16, packet interface{}) error
 	pn.writeMtx.Lock()
 	defer pn.writeMtx.Unlock()
 	if err = pn.conn.WriteMessage(websocket.BinaryMessage, pushMsg); err != nil {
-		gLog.Printf(LevelERROR, "push to %s error:%s", to, err)
+		gLog.Printf(LvERROR, "push to %s error:%s", to, err)
 		pn.conn.Close()
 	}
 	return err
@@ -564,13 +652,13 @@ func (pn *P2PNetwork) read(node string, mainType uint16, subType uint16, timeout
 	for {
 		select {
 		case <-time.After(timeout):
-			gLog.Printf(LevelERROR, "wait msg%d:%d timeout", mainType, subType)
+			gLog.Printf(LvERROR, "wait msg%d:%d timeout", mainType, subType)
 			return
 		case msg := <-ch:
 			head = &openP2PHeader{}
 			err := binary.Read(bytes.NewReader(msg[:openP2PHeaderSize]), binary.LittleEndian, head)
 			if err != nil {
-				gLog.Println(LevelERROR, "read msg error:", err)
+				gLog.Println(LvERROR, "read msg error:", err)
 				break
 			}
 			if head.MainType != mainType || head.SubType != subType {
@@ -596,3 +684,23 @@ func (pn *P2PNetwork) updateAppHeartbeat(appID uint64) {
 		return false
 	})
 }
+
+func (pn *P2PNetwork) refreshIPv6() {
+	if !IsIPv6(pn.config.publicIPv6) { // not support ipv6, not refresh
+		return
+	}
+	client := &http.Client{Timeout: time.Second * 10}
+	r, err := client.Get("http://6.ipw.cn")
+	if err != nil {
+		gLog.Println(LvINFO, "refreshIPv6 error:", err)
+		return
+	}
+	defer r.Body.Close()
+	buf := make([]byte, 1024)
+	n, err := r.Body.Read(buf)
+	if n <= 0 {
+		gLog.Println(LvINFO, "netInfo error:", err, n)
+		return
+	}
+	pn.config.publicIPv6 = string(buf[:n])
+}

p2ptunnel.go

@@ -14,7 +14,7 @@ import (
 
 type P2PTunnel struct {
 	pn            *P2PNetwork
-	conn          p2pConn
+	conn          underlay
 	hbTime        time.Time
 	hbMtx         sync.Mutex
 	hbTimeRelay   time.Time
@@ -25,43 +25,92 @@ type P2PTunnel struct {
 	id            uint64
 	running       bool
 	runMtx        sync.Mutex
-	isServer      bool // 0:server 1:client
+	tunnelServer  bool // different from underlayServer
 	coneLocalPort int
 	coneNatPort   int
+	linkModeWeb   string // use config.linkmode
 }
 
-func (t *P2PTunnel) init() {
+func (t *P2PTunnel) requestPeerInfo() error {
+	// request peer info
+	t.pn.write(MsgQuery, MsgQueryPeerInfoReq, &QueryPeerInfoReq{t.config.peerToken, t.config.PeerNode})
+	head, body := t.pn.read("", MsgQuery, MsgQueryPeerInfoRsp, time.Second*10)
+	if head == nil {
+		return ErrPeerOffline
+	}
+	rsp := QueryPeerInfoRsp{}
+	err := json.Unmarshal(body, &rsp)
+	if err != nil {
+		gLog.Printf(LvERROR, "wrong QueryPeerInfoRsp:%s", err)
+		return ErrMsgFormat
+	}
+	if rsp.Online == 0 {
+		return ErrPeerOffline
+	}
+	if compareVersion(rsp.Version, LeastSupportVersion) == LESS {
+		return ErrVersionNotCompatible
+	}
+	t.config.peerVersion = rsp.Version
+	t.config.hasIPv4 = rsp.HasIPv4
+	t.config.peerIP = rsp.IPv4
+	t.config.peerIPv6 = rsp.IPv6
+	t.config.hasUPNPorNATPMP = rsp.HasUPNPorNATPMP
+	t.config.peerNatType = rsp.NatType
+	///
+	return nil
+}
+func (t *P2PTunnel) initPort() {
 	t.running = true
 	t.hbMtx.Lock()
 	t.hbTime = time.Now()
 	t.hbMtx.Unlock()
 	t.hbTimeRelay = time.Now().Add(time.Second * 600) // TODO: test fake time
-	localPort := int(rand.Uint32()%10000 + 50000)
-	if t.pn.config.natType == NATCone {
-		// prepare one random cone hole
-		_, _, port1, _ := natTest(t.pn.config.ServerHost, t.pn.config.UDPPort1, localPort, 0)
-		t.coneLocalPort = localPort
-		t.coneNatPort = port1
-		t.la = &net.UDPAddr{IP: net.ParseIP(t.pn.config.localIP), Port: t.coneLocalPort}
-	} else {
-		t.coneLocalPort = localPort
-		t.coneNatPort = localPort // NATNONE or symmetric doesn't need coneNatPort
-		t.la = &net.UDPAddr{IP: net.ParseIP(t.pn.config.localIP), Port: t.coneLocalPort}
+	localPort := int(rand.Uint32()%15000 + 50000)     // if the process has bug, will add many upnp port. use specify p2p port by param
+	if t.config.linkMode == LinkModeTCP6 {
+		t.pn.refreshIPv6()
 	}
-	gLog.Printf(LevelDEBUG, "prepare punching port %d:%d", t.coneLocalPort, t.coneNatPort)
+	if t.config.linkMode == LinkModeTCP6 || t.config.linkMode == LinkModeTCP4 {
+		t.coneLocalPort = t.pn.config.TCPPort
+		t.coneNatPort = t.pn.config.TCPPort // symmetric doesn't need coneNatPort
+	}
+	if t.config.linkMode == LinkModeUDPPunch {
+		// prepare one random cone hole
+		_, _, _, natPort, _ := natTest(t.pn.config.ServerHost, t.pn.config.UDPPort1, localPort, 0)
+		t.coneLocalPort = localPort
+		t.coneNatPort = natPort
+	}
+	if t.config.linkMode == LinkModeTCPPunch {
+		// prepare one random cone hole
+		_, natPort := natTCP(t.pn.config.ServerHost, IfconfigPort1, localPort)
+		t.coneLocalPort = localPort
+		t.coneNatPort = natPort
+	}
+	t.la = &net.UDPAddr{IP: net.ParseIP(t.pn.config.localIP), Port: t.coneLocalPort}
+	gLog.Printf(LvDEBUG, "prepare punching port %d:%d", t.coneLocalPort, t.coneNatPort)
 }
 
 func (t *P2PTunnel) connect() error {
-	gLog.Printf(LevelDEBUG, "start p2pTunnel to %s ", t.config.PeerNode)
-	t.isServer = false
+	gLog.Printf(LvDEBUG, "start p2pTunnel to %s ", t.config.PeerNode)
+	t.tunnelServer = false
+	appKey := uint64(0)
 	req := PushConnectReq{
-		Token:       t.config.peerToken,
-		From:        t.pn.config.Node,
-		FromToken:   t.pn.config.Token,
-		FromIP:      t.pn.config.publicIP,
-		ConeNatPort: t.coneNatPort,
-		NatType:     t.pn.config.natType,
-		ID:          t.id}
+		Token:            t.config.peerToken,
+		From:             t.pn.config.Node,
+		FromIP:           t.pn.config.publicIP,
+		ConeNatPort:      t.coneNatPort,
+		NatType:          t.pn.config.natType,
+		HasIPv4:          t.pn.config.hasIPv4,
+		IPv6:             t.pn.config.publicIPv6,
+		HasUPNPorNATPMP:  t.pn.config.hasUPNPorNATPMP,
+		ID:               t.id,
+		AppKey:           appKey,
+		Version:          OpenP2PVersion,
+		LinkMode:         t.config.linkMode,
+		IsUnderlayServer: t.config.isUnderlayServer ^ 1,
+	}
+	if req.Token == 0 { // no relay token
+		req.Token = t.pn.config.Token
+	}
 	t.pn.push(t.config.PeerNode, MsgPushConnectReq, req)
 	head, body := t.pn.read(t.config.PeerNode, MsgPush, MsgPushConnectRsp, time.Second*10)
 	if head == nil {
@@ -70,19 +119,23 @@ func (t *P2PTunnel) connect() error {
 	rsp := PushConnectRsp{}
 	err := json.Unmarshal(body, &rsp)
 	if err != nil {
-		gLog.Printf(LevelERROR, "wrong MsgPushConnectRsp:%s", err)
+		gLog.Printf(LvERROR, "wrong MsgPushConnectRsp:%s", err)
 		return err
 	}
 	// gLog.Println(LevelINFO, rsp)
 	if rsp.Error != 0 {
 		return errors.New(rsp.Detail)
 	}
-	t.config.peerNatType = int(rsp.NatType)
+	t.config.peerNatType = rsp.NatType
+	t.config.hasIPv4 = rsp.HasIPv4
+	t.config.peerIPv6 = rsp.IPv6
+	t.config.hasUPNPorNATPMP = rsp.HasUPNPorNATPMP
+	t.config.peerVersion = rsp.Version
 	t.config.peerConeNatPort = rsp.ConeNatPort
 	t.config.peerIP = rsp.FromIP
-	err = t.handshake()
+	err = t.start()
 	if err != nil {
-		gLog.Println(LevelERROR, "handshake error:", err)
+		gLog.Println(LvERROR, "handshake error:", err)
 		err = ErrorHandshake
 	}
 	return err
@@ -135,18 +188,32 @@ func (t *P2PTunnel) close() {
 	t.pn.allTunnels.Delete(t.id)
 }
 
+func (t *P2PTunnel) start() error {
+	if t.config.linkMode == LinkModeUDPPunch {
+		if err := t.handshake(); err != nil {
+			return err
+		}
+	}
+	err := t.connectUnderlay()
+	if err != nil {
+		gLog.Println(LvERROR, err)
+		return err
+	}
+	return nil
+}
+
 func (t *P2PTunnel) handshake() error {
-	if t.config.peerConeNatPort > 0 {
+	if t.config.peerConeNatPort > 0 { // only peer is cone should prepare t.ra
 		var err error
 		t.ra, err = net.ResolveUDPAddr("udp", fmt.Sprintf("%s:%d", t.config.peerIP, t.config.peerConeNatPort))
 		if err != nil {
 			return err
 		}
 	}
-	gLog.Println(LevelDEBUG, "handshake to ", t.config.PeerNode)
+	gLog.Println(LvDEBUG, "handshake to ", t.config.PeerNode)
 	var err error
 	// TODO: handle NATNone, nodes with public ip has no punching
-	if (t.pn.config.natType == NATCone && t.config.peerNatType == NATCone) || (t.pn.config.natType == NATNone || t.config.peerNatType == NATNone) {
+	if t.pn.config.natType == NATCone && t.config.peerNatType == NATCone {
 		err = handshakeC2C(t)
 	} else if t.config.peerNatType == NATSymmetric && t.pn.config.natType == NATSymmetric {
 		err = ErrorS2S
@@ -159,50 +226,64 @@ func (t *P2PTunnel) handshake() error {
 		return errors.New("unknown error")
 	}
 	if err != nil {
-		gLog.Println(LevelERROR, "punch handshake error:", err)
-		return err
-	}
-	gLog.Printf(LevelDEBUG, "handshake to %s ok", t.config.PeerNode)
-	err = t.run()
-	if err != nil {
-		gLog.Println(LevelERROR, err)
+		gLog.Println(LvERROR, "punch handshake error:", err)
 		return err
 	}
+	gLog.Printf(LvDEBUG, "handshake to %s ok", t.config.PeerNode)
 	return nil
 }
 
-func (t *P2PTunnel) run() error {
-	if t.isServer {
-		qConn, e := listenQuic(t.la.String(), TunnelIdleTimeout)
-		if e != nil {
-			gLog.Println(LevelINFO, "listen quic error:", e, ", retry...")
-			time.Sleep(time.Millisecond * 10)
-			qConn, e = listenQuic(t.la.String(), TunnelIdleTimeout)
-			if e != nil {
-				return fmt.Errorf("listen quic error:%s", e)
-			}
+func (t *P2PTunnel) connectUnderlay() (err error) {
+	switch t.config.linkMode {
+	case LinkModeTCP6:
+		t.conn, err = t.connectUnderlayTCP6()
+	case LinkModeTCP4:
+		t.conn, err = t.connectUnderlayTCP()
+	case LinkModeTCPPunch:
+		t.conn, err = t.connectUnderlayTCP()
+	case LinkModeUDPPunch:
+		t.conn, err = t.connectUnderlayQuic()
+
+	}
+	if err != nil {
+		return err
+	}
+	if t.conn == nil {
+		return errors.New("connect underlay error")
+	}
+	t.setRun(true)
+	go t.readLoop()
+	go t.heartbeatLoop()
+	return nil
+}
+
+func (t *P2PTunnel) connectUnderlayQuic() (c underlay, err error) {
+	gLog.Println(LvINFO, "connectUnderlayQuic start")
+	defer gLog.Println(LvINFO, "connectUnderlayQuic end")
+	var qConn *underlayQUIC
+	if t.config.isUnderlayServer == 1 {
+		time.Sleep(time.Millisecond * 10) // punching udp port will need some times in some env
+		qConn, err = listenQuic(t.la.String(), TunnelIdleTimeout)
+		if err != nil {
+			gLog.Println(LvINFO, "listen quic error:", err, ", retry...")
 		}
-		t.pn.push(t.config.PeerNode, MsgPushQuicConnect, nil)
-		e = qConn.Accept()
-		if e != nil {
+		t.pn.push(t.config.PeerNode, MsgPushUnderlayConnect, nil)
+		err = qConn.Accept()
+		if err != nil {
 			qConn.CloseListener()
-			return fmt.Errorf("accept quic error:%s", e)
+			return nil, fmt.Errorf("accept quic error:%s", err)
 		}
-		_, buff, err := qConn.ReadMessage()
-		if e != nil {
+		_, buff, err := qConn.ReadBuffer()
+		if err != nil {
 			qConn.listener.Close()
-			return fmt.Errorf("read start msg error:%s", err)
+			return nil, fmt.Errorf("read start msg error:%s", err)
 		}
 		if buff != nil {
-			gLog.Println(LevelDEBUG, string(buff))
+			gLog.Println(LvDEBUG, string(buff))
 		}
 		qConn.WriteBytes(MsgP2P, MsgTunnelHandshakeAck, []byte("OpenP2P,hello2"))
-		gLog.Println(LevelDEBUG, "quic connection ok")
-		t.conn = qConn
-		t.setRun(true)
-		go t.readLoop()
-		go t.writeLoop()
-		return nil
+		gLog.Println(LvDEBUG, "quic connection ok")
+		return qConn, nil
 	}
 
 	//else
@@ -211,44 +292,135 @@ func (t *P2PTunnel) run() error {
 		time.Sleep(time.Millisecond * 10)
 		conn, e = net.ListenUDP("udp", t.la)
 		if e != nil {
-			return fmt.Errorf("quic listen error:%s", e)
+			return nil, fmt.Errorf("quic listen error:%s", e)
 		}
 	}
-	t.pn.read(t.config.PeerNode, MsgPush, MsgPushQuicConnect, time.Second*5)
-	gLog.Println(LevelDEBUG, "quic dial to ", t.ra.String())
-	qConn, e := dialQuic(conn, t.ra, TunnelIdleTimeout)
+	t.pn.read(t.config.PeerNode, MsgPush, MsgPushUnderlayConnect, time.Second*5)
+	gLog.Println(LvDEBUG, "quic dial to ", t.ra.String())
+	qConn, e = dialQuic(conn, t.ra, TunnelIdleTimeout)
 	if e != nil {
-		return fmt.Errorf("quic dial to %s error:%s", t.ra.String(), e)
+		return nil, fmt.Errorf("quic dial to %s error:%s", t.ra.String(), e)
 	}
 	handshakeBegin := time.Now()
 	qConn.WriteBytes(MsgP2P, MsgTunnelHandshake, []byte("OpenP2P,hello"))
-	_, buff, err := qConn.ReadMessage()
+	_, buff, err := qConn.ReadBuffer()
 	if e != nil {
 		qConn.listener.Close()
-		return fmt.Errorf("read MsgTunnelHandshake error:%s", err)
+		return nil, fmt.Errorf("read MsgTunnelHandshake error:%s", err)
 	}
 	if buff != nil {
-		gLog.Println(LevelDEBUG, string(buff))
+		gLog.Println(LvDEBUG, string(buff))
 	}
 
-	gLog.Println(LevelINFO, "rtt=", time.Since(handshakeBegin))
-	gLog.Println(LevelDEBUG, "quic connection ok")
-	t.conn = qConn
-	t.setRun(true)
-	go t.readLoop()
-	go t.writeLoop()
-	return nil
+	gLog.Println(LvINFO, "rtt=", time.Since(handshakeBegin))
+	gLog.Println(LvDEBUG, "quic connection ok")
+	t.linkModeWeb = LinkModeUDPPunch
+	return qConn, nil
+}
+
+// websocket
+func (t *P2PTunnel) connectUnderlayTCP() (c underlay, err error) {
+	gLog.Println(LvINFO, "connectUnderlayTCP start")
+	defer gLog.Println(LvINFO, "connectUnderlayTCP end")
+	var qConn *underlayTCP
+	if t.config.isUnderlayServer == 1 {
+		t.pn.push(t.config.PeerNode, MsgPushUnderlayConnect, nil)
+		qConn, err = listenTCP(t.config.peerIP, t.config.peerConeNatPort, t.coneLocalPort, t.config.linkMode)
+		if err != nil {
+			return nil, fmt.Errorf("listen TCP error:%s", err)
+		}
+
+		_, buff, err := qConn.ReadBuffer()
+		if err != nil {
+			return nil, fmt.Errorf("read start msg error:%s", err)
+		}
+		if buff != nil {
+			gLog.Println(LvDEBUG, string(buff))
+		}
+		qConn.WriteBytes(MsgP2P, MsgTunnelHandshakeAck, []byte("OpenP2P,hello2"))
+		gLog.Println(LvINFO, "TCP connection ok")
+		return qConn, nil
+	}
+
+	//else
+	t.pn.read(t.config.PeerNode, MsgPush, MsgPushUnderlayConnect, time.Second*5)
+	gLog.Println(LvDEBUG, "TCP dial to ", t.config.peerIP, ":", t.config.peerConeNatPort)
+	qConn, err = dialTCP(t.config.peerIP, t.config.peerConeNatPort, t.coneLocalPort, t.config.linkMode)
+	if err != nil {
+		return nil, fmt.Errorf("TCP dial to %s:%d error:%s", t.config.peerIP, t.config.peerConeNatPort, err)
+	}
+	handshakeBegin := time.Now()
+	qConn.WriteBytes(MsgP2P, MsgTunnelHandshake, []byte("OpenP2P,hello"))
+	_, buff, err := qConn.ReadBuffer()
+	if err != nil {
+		return nil, fmt.Errorf("read MsgTunnelHandshake error:%s", err)
+	}
+	if buff != nil {
+		gLog.Println(LvDEBUG, string(buff))
+	}
+
+	gLog.Println(LvINFO, "rtt=", time.Since(handshakeBegin))
+	gLog.Println(LvINFO, "TCP connection ok")
+	t.linkModeWeb = LinkModeIPv4
+	return qConn, nil
+}
+
+func (t *P2PTunnel) connectUnderlayTCP6() (c underlay, err error) {
+	gLog.Println(LvINFO, "connectUnderlayTCP6 start")
+	defer gLog.Println(LvINFO, "connectUnderlayTCP6 end")
+	var qConn *underlayTCP6
+	if t.config.isUnderlayServer == 1 {
+		t.pn.push(t.config.PeerNode, MsgPushUnderlayConnect, nil)
+		qConn, err = listenTCP6(t.coneNatPort, TunnelIdleTimeout)
+		if err != nil {
+			return nil, fmt.Errorf("listen TCP6 error:%s", err)
+		}
+		_, buff, err := qConn.ReadBuffer()
+		if err != nil {
+			qConn.listener.Close()
+			return nil, fmt.Errorf("read start msg error:%s", err)
+		}
+		if buff != nil {
+			gLog.Println(LvDEBUG, string(buff))
+		}
+		qConn.WriteBytes(MsgP2P, MsgTunnelHandshakeAck, []byte("OpenP2P,hello2"))
+		gLog.Println(LvDEBUG, "TCP6 connection ok")
+		return qConn, nil
+	}
+
+	//else
+	t.pn.read(t.config.PeerNode, MsgPush, MsgPushUnderlayConnect, time.Second*5)
+	gLog.Println(LvDEBUG, "TCP6 dial to ", t.config.peerIPv6)
+	qConn, err = dialTCP6(t.config.peerIPv6, t.config.peerConeNatPort)
+	if err != nil {
+		return nil, fmt.Errorf("TCP6 dial to %s:%d error:%s", t.config.peerIPv6, t.config.peerConeNatPort, err)
+	}
+	handshakeBegin := time.Now()
+	qConn.WriteBytes(MsgP2P, MsgTunnelHandshake, []byte("OpenP2P,hello"))
+	_, buff, err := qConn.ReadBuffer()
+	if err != nil {
+		qConn.listener.Close()
+		return nil, fmt.Errorf("read MsgTunnelHandshake error:%s", err)
+	}
+	if buff != nil {
+		gLog.Println(LvDEBUG, string(buff))
+	}
+
+	gLog.Println(LvINFO, "rtt=", time.Since(handshakeBegin))
+	gLog.Println(LvDEBUG, "TCP6 connection ok")
+	t.linkModeWeb = LinkModeIPv6
+	return qConn, nil
 }
 
 func (t *P2PTunnel) readLoop() {
 	decryptData := make([]byte, ReadBuffLen+PaddingSize) // 16 bytes for padding
-	gLog.Printf(LevelDEBUG, "%d tunnel readloop start", t.id)
+	gLog.Printf(LvDEBUG, "%d tunnel readloop start", t.id)
 	for t.isRuning() {
 		t.conn.SetReadDeadline(time.Now().Add(TunnelIdleTimeout))
-		head, body, err := t.conn.ReadMessage()
+		head, body, err := t.conn.ReadBuffer()
 		if err != nil {
 			if t.isRuning() {
-				gLog.Printf(LevelERROR, "%d tunnel read error:%s", t.id, err)
+				gLog.Printf(LvERROR, "%d tunnel read error:%s", t.id, err)
 			}
 			break
 		}
@@ -258,25 +430,25 @@ func (t *P2PTunnel) readLoop() {
 		switch head.SubType {
 		case MsgTunnelHeartbeat:
 			t.conn.WriteBytes(MsgP2P, MsgTunnelHeartbeatAck, nil)
-			gLog.Printf(LevelDEBUG, "%d read tunnel heartbeat", t.id)
+			gLog.Printf(LvDEBUG, "%d read tunnel heartbeat", t.id)
 		case MsgTunnelHeartbeatAck:
 			t.hbMtx.Lock()
 			t.hbTime = time.Now()
 			t.hbMtx.Unlock()
-			gLog.Printf(LevelDEBUG, "%d read tunnel heartbeat ack", t.id)
+			gLog.Printf(LvDEBUG, "%d read tunnel heartbeat ack", t.id)
 		case MsgOverlayData:
 			if len(body) < overlayHeaderSize {
 				continue
 			}
 			overlayID := binary.LittleEndian.Uint64(body[:8])
-			gLog.Printf(LevelDEBUG, "%d tunnel read overlay data %d", t.id, overlayID)
+			gLog.Printf(LvDEBUG, "%d tunnel read overlay data %d", t.id, overlayID)
 			s, ok := t.overlayConns.Load(overlayID)
 			if !ok {
 				// debug level, when overlay connection closed, always has some packet not found tunnel
-				gLog.Printf(LevelDEBUG, "%d tunnel not found overlay connection %d", t.id, overlayID)
+				gLog.Printf(LvDEBUG, "%d tunnel not found overlay connection %d", t.id, overlayID)
 				continue
 			}
-			overlayConn, ok := s.(*overlayTCP)
+			overlayConn, ok := s.(*overlayConn)
 			if !ok {
 				continue
 			}
@@ -287,10 +459,10 @@ func (t *P2PTunnel) readLoop() {
 			}
 			_, err = overlayConn.Write(payload)
 			if err != nil {
-				gLog.Println(LevelERROR, "overlay write error:", err)
+				gLog.Println(LvERROR, "overlay write error:", err)
 			}
 		case MsgRelayData:
-			gLog.Printf(LevelDEBUG, "got relay data datalen=%d", head.DataLen)
+			gLog.Printf(LvDEBUG, "got relay data datalen=%d", head.DataLen)
 			if len(body) < 8 {
 				continue
 			}
@@ -300,10 +472,10 @@ func (t *P2PTunnel) readLoop() {
 			req := RelayHeartbeat{}
 			err := json.Unmarshal(body, &req)
 			if err != nil {
-				gLog.Printf(LevelERROR, "wrong RelayHeartbeat:%s", err)
+				gLog.Printf(LvERROR, "wrong RelayHeartbeat:%s", err)
 				continue
 			}
-			gLog.Printf(LevelDEBUG, "got MsgRelayHeartbeat from %d:%d", req.RelayTunnelID, req.AppID)
+			gLog.Printf(LvDEBUG, "got MsgRelayHeartbeat from %d:%d", req.RelayTunnelID, req.AppID)
 			relayHead := new(bytes.Buffer)
 			binary.Write(relayHead, binary.LittleEndian, req.RelayTunnelID)
 			msg, _ := newMessage(MsgP2P, MsgRelayHeartbeatAck, &req)
@@ -313,104 +485,136 @@ func (t *P2PTunnel) readLoop() {
 			req := RelayHeartbeat{}
 			err := json.Unmarshal(body, &req)
 			if err != nil {
-				gLog.Printf(LevelERROR, "wrong RelayHeartbeat:%s", err)
+				gLog.Printf(LvERROR, "wrong RelayHeartbeat:%s", err)
 				continue
 			}
-			gLog.Printf(LevelDEBUG, "got MsgRelayHeartbeatAck to %d", req.AppID)
+			gLog.Printf(LvDEBUG, "got MsgRelayHeartbeatAck to %d", req.AppID)
 			t.pn.updateAppHeartbeat(req.AppID)
 		case MsgOverlayConnectReq:
 			req := OverlayConnectReq{}
 			err := json.Unmarshal(body, &req)
 			if err != nil {
-				gLog.Printf(LevelERROR, "wrong MsgOverlayConnectReq:%s", err)
+				gLog.Printf(LvERROR, "wrong MsgOverlayConnectReq:%s", err)
 				continue
 			}
 			// app connect only accept token(not relay totp token), avoid someone using the share relay node's token
 			if req.Token != t.pn.config.Token {
-				gLog.Println(LevelERROR, "Access Denied:", req.Token)
+				gLog.Println(LvERROR, "Access Denied:", req.Token)
 				continue
 			}
 
 			overlayID := req.ID
-			gLog.Printf(LevelDEBUG, "App:%d overlayID:%d connect %+v", req.AppID, overlayID, req)
-			if req.Protocol == "tcp" {
-				conn, err := net.DialTimeout("tcp", fmt.Sprintf("%s:%d", req.DstIP, req.DstPort), time.Second*5)
-				if err != nil {
-					gLog.Println(LevelERROR, err)
-					continue
-				}
-				otcp := overlayTCP{
-					tunnel:   t,
-					conn:     conn,
-					id:       overlayID,
-					isClient: false,
-					rtid:     req.RelayTunnelID,
-					appID:    req.AppID,
-					appKey:   GetKey(req.AppID),
-				}
-				// calc key bytes for encrypt
-				if otcp.appKey != 0 {
-					encryptKey := make([]byte, 16)
-					binary.LittleEndian.PutUint64(encryptKey, otcp.appKey)
-					binary.LittleEndian.PutUint64(encryptKey[8:], otcp.appKey)
-					otcp.appKeyBytes = encryptKey
-				}
-
-				t.overlayConns.Store(otcp.id, &otcp)
-				go otcp.run()
+			gLog.Printf(LvDEBUG, "App:%d overlayID:%d connect %+v", req.AppID, overlayID, req)
+			oConn := overlayConn{
+				tunnel:   t,
+				id:       overlayID,
+				isClient: false,
+				rtid:     req.RelayTunnelID,
+				appID:    req.AppID,
+				appKey:   GetKey(req.AppID),
 			}
+			if req.Protocol == "udp" {
+				oConn.connUDP, err = net.DialUDP("udp", nil, &net.UDPAddr{IP: net.ParseIP(req.DstIP), Port: req.DstPort})
+			} else {
+				oConn.connTCP, err = net.DialTimeout("tcp", fmt.Sprintf("%s:%d", req.DstIP, req.DstPort), time.Second*5)
+			}
+			if err != nil {
+				gLog.Println(LvERROR, err)
+				continue
+			}
+
+			// calc key bytes for encrypt
+			if oConn.appKey != 0 {
+				encryptKey := make([]byte, 16)
+				binary.LittleEndian.PutUint64(encryptKey, oConn.appKey)
+				binary.LittleEndian.PutUint64(encryptKey[8:], oConn.appKey)
+				oConn.appKeyBytes = encryptKey
+			}
+
+			t.overlayConns.Store(oConn.id, &oConn)
+			go oConn.run()
 		case MsgOverlayDisconnectReq:
 			req := OverlayDisconnectReq{}
 			err := json.Unmarshal(body, &req)
 			if err != nil {
-				gLog.Printf(LevelERROR, "wrong OverlayDisconnectRequest:%s", err)
+				gLog.Printf(LvERROR, "wrong OverlayDisconnectRequest:%s", err)
 				continue
 			}
 			overlayID := req.ID
-			gLog.Printf(LevelDEBUG, "%d disconnect overlay connection %d", t.id, overlayID)
+			gLog.Printf(LvDEBUG, "%d disconnect overlay connection %d", t.id, overlayID)
 			i, ok := t.overlayConns.Load(overlayID)
 			if ok {
-				otcp := i.(*overlayTCP)
-				otcp.running = false
+				oConn := i.(*overlayConn)
+				oConn.running = false
 			}
 		default:
 		}
 	}
 	t.setRun(false)
 	t.conn.Close()
-	gLog.Printf(LevelDEBUG, "%d tunnel readloop end", t.id)
+	gLog.Printf(LvDEBUG, "%d tunnel readloop end", t.id)
 }
 
-func (t *P2PTunnel) writeLoop() {
+func (t *P2PTunnel) heartbeatLoop() {
 	tc := time.NewTicker(TunnelHeartbeatTime)
 	defer tc.Stop()
-	defer gLog.Printf(LevelDEBUG, "%d tunnel writeloop end", t.id)
+	gLog.Printf(LvDEBUG, "%d tunnel heartbeatLoop start", t.id)
+	defer gLog.Printf(LvDEBUG, "%d tunnel heartbeatLoop end", t.id)
 	for t.isRuning() {
 		select {
 		case <-tc.C:
 			// tunnel send
 			err := t.conn.WriteBytes(MsgP2P, MsgTunnelHeartbeat, nil)
 			if err != nil {
-				gLog.Printf(LevelERROR, "%d write tunnel heartbeat error %s", t.id, err)
+				gLog.Printf(LvERROR, "%d write tunnel heartbeat error %s", t.id, err)
 				t.setRun(false)
 				return
 			}
-			gLog.Printf(LevelDEBUG, "%d write tunnel heartbeat ok", t.id)
+			gLog.Printf(LvDEBUG, "%d write tunnel heartbeat ok", t.id)
 		}
 	}
 }
 
 func (t *P2PTunnel) listen() error {
-	gLog.Printf(LevelDEBUG, "p2ptunnel wait for connecting")
-	t.isServer = true
-	return t.handshake()
+	// notify client to connect
+	rsp := PushConnectRsp{
+		Error:   0,
+		Detail:  "connect ok",
+		To:      t.config.PeerNode,
+		From:    t.pn.config.Node,
+		NatType: t.pn.config.natType,
+		HasIPv4: t.pn.config.hasIPv4,
+		// IPv6:            t.pn.config.IPv6,
+		HasUPNPorNATPMP: t.pn.config.hasUPNPorNATPMP,
+		FromIP:          t.pn.config.publicIP,
+		ConeNatPort:     t.coneNatPort,
+		ID:              t.id,
+		Version:         OpenP2PVersion,
+	}
+	// only private node set ipv6
+	if t.config.fromToken == t.pn.config.Token {
+		t.pn.refreshIPv6()
+		rsp.IPv6 = t.pn.config.publicIPv6
+	}
+
+	t.pn.push(t.config.PeerNode, MsgPushConnectRsp, rsp)
+	gLog.Printf(LvDEBUG, "p2ptunnel wait for connecting")
+	t.tunnelServer = true
+	return t.start()
 }
 
 func (t *P2PTunnel) closeOverlayConns(appID uint64) {
 	t.overlayConns.Range(func(_, i interface{}) bool {
-		otcp := i.(*overlayTCP)
-		if otcp.appID == appID {
-			otcp.conn.Close()
+		oConn := i.(*overlayConn)
+		if oConn.appID == appID {
+			if oConn.connTCP != nil {
+				oConn.connTCP.Close()
+				oConn.connTCP = nil
+			}
+			if oConn.connUDP != nil {
+				oConn.connUDP.Close()
+				oConn.connUDP = nil
+			}
 		}
 		return true
 	})

protocol.go

@@ -10,8 +10,17 @@ import (
 	"time"
 )
 
-const OpenP2PVersion = "1.1.0"
+const OpenP2PVersion = "3.2.0"
 const ProducnName string = "openp2p"
+const LeastSupportVersion = "3.0.0"
+
+const (
+	IfconfigPort1 = 27180
+	IfconfigPort2 = 27181
+	WsPort        = 27183
+	UDPPort1      = 27182
+	UDPPort2      = 27183
+)
 
 type openP2PHeader struct {
 	DataLen  uint32
@@ -67,8 +76,10 @@ const (
 	MsgP2P       = 4
 	MsgRelay     = 5
 	MsgReport    = 6
+	MsgQuery     = 7
 )
 
+// TODO: seperate node push and web push.
 const (
 	MsgPushRsp               = 0
 	MsgPushConnectReq        = 1
@@ -78,11 +89,13 @@ const (
 	MsgPushAddRelayTunnelRsp = 5
 	MsgPushUpdate            = 6
 	MsgPushReportApps        = 7
-	MsgPushQuicConnect       = 8
+	MsgPushUnderlayConnect   = 8
 	MsgPushEditApp           = 9
 	MsgPushSwitchApp         = 10
 	MsgPushRestart           = 11
 	MsgPushEditNode          = 12
+	MsgPushAPPKey            = 13
+	MsgPushReportLog         = 14
 )
 
 // MsgP2P sub type message
@@ -114,10 +127,11 @@ const (
 	MsgReportQuery
 	MsgReportConnect
 	MsgReportApps
+	MsgReportLog
 )
 
 const (
-	ReadBuffLen           = 1024
+	ReadBuffLen           = 4096             // for UDP maybe not enough
 	NetworkHeartbeatTime  = time.Second * 30 // TODO: server no response hb, save flow
 	TunnelHeartbeatTime   = time.Second * 15
 	TunnelIdleTimeout     = time.Minute
@@ -131,9 +145,10 @@ const (
 	AESKeySize                   = 16
 	MaxRetry                     = 10
 	RetryInterval                = time.Second * 30
-	PublicIPEchoTimeout          = time.Second * 3
+	PublicIPEchoTimeout          = time.Second * 1
 	NatTestTimeout               = time.Second * 10
 	ClientAPITimeout             = time.Second * 10
+	MaxDirectTry                 = 3
 )
 
 // NATNone has public ip
@@ -144,6 +159,30 @@ const (
 	NATUnknown   = 314
 )
 
+// underlay protocol
+const (
+	UderlayAuto = "auto"
+	UderlayQUIC = "quic"
+	UderlayTCP  = "tcp"
+)
+
+// linkmode
+const (
+	LinkModeUDPPunch = "udppunch"
+	LinkModeTCPPunch = "tcppunch"
+	LinkModeIPv4     = "ipv4" // for web
+	LinkModeIPv6     = "ipv6" // for web
+	LinkModeTCP6     = "tcp6"
+	LinkModeTCP4     = "tcp4"
+	LinkModeUDP6     = "udp6"
+	LinkModeUDP4     = "udp4"
+)
+
+const (
+	MsgQueryPeerInfoReq = iota
+	MsgQueryPeerInfoRsp
+)
+
 func newMessage(mainType uint16, subType uint16, packet interface{}) ([]byte, error) {
 	data, err := json.Marshal(packet)
 	if err != nil {
@@ -169,23 +208,34 @@ func nodeNameToID(name string) uint64 {
 }
 
 type PushConnectReq struct {
-	From        string `json:"from,omitempty"`
-	FromToken   uint64 `json:"fromToken,omitempty"` //my token
-	Token       uint64 `json:"token,omitempty"`     // totp token
-	ConeNatPort int    `json:"coneNatPort,omitempty"`
-	NatType     int    `json:"natType,omitempty"`
-	FromIP      string `json:"fromIP,omitempty"`
-	ID          uint64 `json:"id,omitempty"`
+	From             string `json:"from,omitempty"`
+	FromToken        uint64 `json:"fromToken,omitempty"` // deprecated
+	Version          string `json:"version,omitempty"`
+	Token            uint64 `json:"token,omitempty"`       // if public totp token
+	ConeNatPort      int    `json:"coneNatPort,omitempty"` // if isPublic, is public port
+	NatType          int    `json:"natType,omitempty"`
+	HasIPv4          int    `json:"hasIPv4,omitempty"`
+	IPv6             string `json:"IPv6,omitempty"`
+	HasUPNPorNATPMP  int    `json:"hasUPNPorNATPMP,omitempty"`
+	FromIP           string `json:"fromIP,omitempty"`
+	ID               uint64 `json:"id,omitempty"`
+	AppKey           uint64 `json:"appKey,omitempty"` // for underlay tcp
+	LinkMode         string `json:"linkMode,omitempty"`
+	IsUnderlayServer int    `json:"isServer,omitempty"` // Requset spec peer is server
 }
 type PushConnectRsp struct {
-	Error       int    `json:"error,omitempty"`
-	From        string `json:"from,omitempty"`
-	To          string `json:"to,omitempty"`
-	Detail      string `json:"detail,omitempty"`
-	NatType     int    `json:"natType,omitempty"`
-	ConeNatPort int    `json:"coneNatPort,omitempty"`
-	FromIP      string `json:"fromIP,omitempty"`
-	ID          uint64 `json:"id,omitempty"`
+	Error           int    `json:"error,omitempty"`
+	From            string `json:"from,omitempty"`
+	To              string `json:"to,omitempty"`
+	Detail          string `json:"detail,omitempty"`
+	NatType         int    `json:"natType,omitempty"`
+	HasIPv4         int    `json:"hasIPv4,omitempty"`
+	IPv6            string `json:"IPv6,omitempty"` // if public relay node, ipv6 not set
+	HasUPNPorNATPMP int    `json:"hasUPNPorNATPMP,omitempty"`
+	ConeNatPort     int    `json:"coneNatPort,omitempty"` //it's not only cone, but also upnp or nat-pmp hole
+	FromIP          string `json:"fromIP,omitempty"`
+	ID              uint64 `json:"id,omitempty"`
+	Version         string `json:"version,omitempty"`
 }
 type PushRsp struct {
 	Error  int    `json:"error,omitempty"`
@@ -236,6 +286,7 @@ type RelayNodeReq struct {
 }
 
 type RelayNodeRsp struct {
+	Mode       string `json:"mode,omitempty"` // private,public
 	RelayName  string `json:"relayName,omitempty"`
 	RelayToken uint64 `json:"relayToken,omitempty"`
 }
@@ -244,8 +295,13 @@ type AddRelayTunnelReq struct {
 	From       string `json:"from,omitempty"`
 	RelayName  string `json:"relayName,omitempty"`
 	RelayToken uint64 `json:"relayToken,omitempty"`
-	AppID      uint64 `json:"appID,omitempty"`
-	AppKey     uint64 `json:"appKey,omitempty"`
+	AppID      uint64 `json:"appID,omitempty"`  // deprecated
+	AppKey     uint64 `json:"appKey,omitempty"` // deprecated
+}
+
+type APPKeySync struct {
+	AppID  uint64 `json:"appID,omitempty"`
+	AppKey uint64 `json:"appKey,omitempty"`
 }
 
 type RelayHeartbeat struct {
@@ -254,12 +310,14 @@ type RelayHeartbeat struct {
 }
 
 type ReportBasic struct {
-	OS      string  `json:"os,omitempty"`
-	Mac     string  `json:"mac,omitempty"`
-	LanIP   string  `json:"lanIP,omitempty"`
-	IPv6    string  `json:"IPv6,omitempty"`
-	Version string  `json:"version,omitempty"`
-	NetInfo NetInfo `json:"netInfo,omitempty"`
+	OS              string  `json:"os,omitempty"`
+	Mac             string  `json:"mac,omitempty"`
+	LanIP           string  `json:"lanIP,omitempty"`
+	HasIPv4         int     `json:"hasIPv4,omitempty"`
+	IPv6            string  `json:"IPv6,omitempty"`
+	HasUPNPorNATPMP int     `json:"hasUPNPorNATPMP,omitempty"`
+	Version         string  `json:"version,omitempty"`
+	NetInfo         NetInfo `json:"netInfo,omitempty"`
 }
 
 type ReportConnect struct {
@@ -269,7 +327,7 @@ type ReportConnect struct {
 	NatType        int    `json:"natType,omitempty"`
 	PeerNode       string `json:"peerNode,omitempty"`
 	DstPort        int    `json:"dstPort,omitempty"`
-	DstHost        string `json:"dsdtHost,omitempty"`
+	DstHost        string `json:"dstHost,omitempty"`
 	PeerUser       string `json:"peerUser,omitempty"`
 	PeerNatType    int    `json:"peerNatType,omitempty"`
 	PeerIP         string `json:"peerIP,omitempty"`
@@ -294,8 +352,11 @@ type AppInfo struct {
 	PeerIP         string `json:"peerIP,omitempty"`
 	ShareBandwidth int    `json:"shareBandWidth,omitempty"`
 	RelayNode      string `json:"relayNode,omitempty"`
+	RelayMode      string `json:"relayMode,omitempty"`
+	LinkMode       string `json:"linkMode,omitempty"`
 	Version        string `json:"version,omitempty"`
 	RetryTime      string `json:"retryTime,omitempty"`
+	ConnectTime    string `json:"connectTime,omitempty"`
 	IsActive       int    `json:"isActive,omitempty"`
 	Enabled        int    `json:"enabled,omitempty"`
 }
@@ -304,6 +365,18 @@ type ReportApps struct {
 	Apps []AppInfo
 }
 
+type ReportLogReq struct {
+	FileName string `json:"fileName,omitempty"`
+	Offset   int64  `json:"offset,omitempty"`
+	Len      int64  `json:"len,omitempty"`
+}
+type ReportLogRsp struct {
+	FileName string `json:"fileName,omitempty"`
+	Content  string `json:"content,omitempty"`
+	Len      int64  `json:"len,omitempty"`
+	Total    int64  `json:"total,omitempty"`
+}
+
 type UpdateInfo struct {
 	Error       int    `json:"error,omitempty"`
 	ErrorDetail string `json:"errorDetail,omitempty"`
@@ -342,3 +415,17 @@ type EditNode struct {
 	NewName   string `json:"newName,omitempty"`
 	Bandwidth int    `json:"bandwidth,omitempty"`
 }
+
+type QueryPeerInfoReq struct {
+	Token    uint64 `json:"token,omitempty"` // if public totp token
+	PeerNode string `json:"peerNode,omitempty"`
+}
+type QueryPeerInfoRsp struct {
+	Online          int    `json:"online,omitempty"`
+	Version         string `json:"version,omitempty"`
+	NatType         int    `json:"natType,omitempty"`
+	IPv4            string `json:"IPv4,omitempty"`
+	HasIPv4         int    `json:"hasIPv4,omitempty"` // has public ipv4
+	IPv6            string `json:"IPv6,omitempty"`    // if public relay node, ipv6 not set
+	HasUPNPorNATPMP int    `json:"hasUPNPorNATPMP,omitempty"`
+}

totp.go

@@ -25,6 +25,9 @@ func VerifyTOTP(code uint64, token uint64, ts int64) bool {
 	if code == 0 {
 		return false
 	}
+	if code == token {
+		return true
+	}
 	if code == GenTOTP(token, ts) || code == GenTOTP(token, ts-TOTPStep) || code == GenTOTP(token, ts+TOTPStep) {
 		return true
 	}

udp.go

@@ -23,7 +23,7 @@ func UDPRead(conn *net.UDPConn, timeout int) (ra net.Addr, head *openP2PHeader,
 		deadline := time.Now().Add(time.Millisecond * time.Duration(timeout))
 		err = conn.SetReadDeadline(deadline)
 		if err != nil {
-			gLog.Println(LevelERROR, "SetReadDeadline error")
+			gLog.Println(LvERROR, "SetReadDeadline error")
 			return nil, nil, nil, 0, err
 		}
 	}
@@ -37,7 +37,7 @@ func UDPRead(conn *net.UDPConn, timeout int) (ra net.Addr, head *openP2PHeader,
 	head = &openP2PHeader{}
 	err = binary.Read(bytes.NewReader(result[:openP2PHeaderSize]), binary.LittleEndian, head)
 	if err != nil {
-		gLog.Println(LevelERROR, "parse p2pheader error:", err)
+		gLog.Println(LvERROR, "parse p2pheader error:", err)
 		return nil, nil, nil, 0, err
 	}
 	return

underlay.go

@@ -0,0 +1,16 @@
+package main
+
+import (
+	"time"
+)
+
+type underlay interface {
+	ReadBuffer() (*openP2PHeader, []byte, error)
+	WriteBytes(uint16, uint16, []byte) error
+	WriteBuffer([]byte) error
+	WriteMessage(uint16, uint16, interface{}) error
+	Close() error
+	SetReadDeadline(t time.Time) error
+	SetWriteDeadline(t time.Time) error
+	Protocol() string
+}

underlay_quic.go

@@ -21,14 +21,18 @@ import (
 //quic.DialContext do not support version 44,disable it
 var quicVersion []quic.VersionNumber
 
-type quicConn struct {
+type underlayQUIC struct {
 	listener quic.Listener
 	writeMtx *sync.Mutex
 	quic.Stream
-	quic.Session
+	quic.Connection
 }
 
-func (conn *quicConn) ReadMessage() (*openP2PHeader, []byte, error) {
+func (conn *underlayQUIC) Protocol() string {
+	return "quic"
+}
+
+func (conn *underlayQUIC) ReadBuffer() (*openP2PHeader, []byte, error) {
 	headBuf := make([]byte, openP2PHeaderSize)
 	_, err := io.ReadFull(conn, headBuf)
 	if err != nil {
@@ -43,7 +47,7 @@ func (conn *quicConn) ReadMessage() (*openP2PHeader, []byte, error) {
 	return head, dataBuf, err
 }
 
-func (conn *quicConn) WriteBytes(mainType uint16, subType uint16, data []byte) error {
+func (conn *underlayQUIC) WriteBytes(mainType uint16, subType uint16, data []byte) error {
 	writeBytes := append(encodeHeader(mainType, subType, uint32(len(data))), data...)
 	conn.writeMtx.Lock()
 	_, err := conn.Write(writeBytes)
@@ -51,14 +55,14 @@ func (conn *quicConn) WriteBytes(mainType uint16, subType uint16, data []byte) e
 	return err
 }
 
-func (conn *quicConn) WriteBuffer(data []byte) error {
+func (conn *underlayQUIC) WriteBuffer(data []byte) error {
 	conn.writeMtx.Lock()
 	_, err := conn.Write(data)
 	conn.writeMtx.Unlock()
 	return err
 }
 
-func (conn *quicConn) WriteMessage(mainType uint16, subType uint16, packet interface{}) error {
+func (conn *underlayQUIC) WriteMessage(mainType uint16, subType uint16, packet interface{}) error {
 	// TODO: call newMessage
 	data, err := json.Marshal(packet)
 	if err != nil {
@@ -71,18 +75,18 @@ func (conn *quicConn) WriteMessage(mainType uint16, subType uint16, packet inter
 	return err
 }
 
-func (conn *quicConn) Close() error {
+func (conn *underlayQUIC) Close() error {
 	conn.Stream.CancelRead(1)
-	conn.Session.CloseWithError(0, "")
+	conn.Connection.CloseWithError(0, "")
 	return nil
 }
-func (conn *quicConn) CloseListener() {
+func (conn *underlayQUIC) CloseListener() {
 	if conn.listener != nil {
 		conn.listener.Close()
 	}
 }
 
-func (conn *quicConn) Accept() error {
+func (conn *underlayQUIC) Accept() error {
 	ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
 	defer cancel()
 	sess, err := conn.listener.Accept(ctx)
@@ -94,35 +98,35 @@ func (conn *quicConn) Accept() error {
 		return err
 	}
 	conn.Stream = stream
-	conn.Session = sess
+	conn.Connection = sess
 	return nil
 }
 
-func listenQuic(addr string, idleTimeout time.Duration) (*quicConn, error) {
-	gLog.Println(LevelDEBUG, "quic listen on ", addr)
+func listenQuic(addr string, idleTimeout time.Duration) (*underlayQUIC, error) {
+	gLog.Println(LvDEBUG, "quic listen on ", addr)
 	listener, err := quic.ListenAddr(addr, generateTLSConfig(),
 		&quic.Config{Versions: quicVersion, MaxIdleTimeout: idleTimeout, DisablePathMTUDiscovery: true})
 	if err != nil {
 		return nil, fmt.Errorf("quic.ListenAddr error:%s", err)
 	}
-	return &quicConn{listener: listener, writeMtx: &sync.Mutex{}}, nil
+	return &underlayQUIC{listener: listener, writeMtx: &sync.Mutex{}}, nil
 }
 
-func dialQuic(conn *net.UDPConn, remoteAddr *net.UDPAddr, idleTimeout time.Duration) (*quicConn, error) {
+func dialQuic(conn *net.UDPConn, remoteAddr *net.UDPAddr, idleTimeout time.Duration) (*underlayQUIC, error) {
 	tlsConf := &tls.Config{
 		InsecureSkipVerify: true,
 		NextProtos:         []string{"openp2pv1"},
 	}
-	session, err := quic.DialContext(context.Background(), conn, remoteAddr, conn.LocalAddr().String(), tlsConf,
+	Connection, err := quic.DialContext(context.Background(), conn, remoteAddr, conn.LocalAddr().String(), tlsConf,
 		&quic.Config{Versions: quicVersion, MaxIdleTimeout: idleTimeout, DisablePathMTUDiscovery: true})
 	if err != nil {
 		return nil, fmt.Errorf("quic.DialContext error:%s", err)
 	}
-	stream, err := session.OpenStreamSync(context.Background())
+	stream, err := Connection.OpenStreamSync(context.Background())
 	if err != nil {
 		return nil, fmt.Errorf("OpenStreamSync error:%s", err)
 	}
-	qConn := &quicConn{nil, &sync.Mutex{}, stream, session}
+	qConn := &underlayQUIC{nil, &sync.Mutex{}, stream, Connection}
 	return qConn, nil
 }
 

underlay_tcp.go

@@ -0,0 +1,108 @@
+package main
+
+import (
+	"encoding/json"
+	"fmt"
+	"io"
+	"net"
+	"sync"
+	"time"
+
+	reuse "github.com/openp2p-cn/go-reuseport"
+)
+
+type underlayTCP struct {
+	writeMtx *sync.Mutex
+	net.Conn
+}
+
+func (conn *underlayTCP) Protocol() string {
+	return "tcp"
+}
+
+func (conn *underlayTCP) ReadBuffer() (*openP2PHeader, []byte, error) {
+	headBuf := make([]byte, openP2PHeaderSize)
+	_, err := io.ReadFull(conn, headBuf)
+	if err != nil {
+		return nil, nil, err
+	}
+	head, err := decodeHeader(headBuf)
+	if err != nil {
+		return nil, nil, err
+	}
+	dataBuf := make([]byte, head.DataLen)
+	_, err = io.ReadFull(conn, dataBuf)
+	return head, dataBuf, err
+}
+
+func (conn *underlayTCP) WriteBytes(mainType uint16, subType uint16, data []byte) error {
+	writeBytes := append(encodeHeader(mainType, subType, uint32(len(data))), data...)
+	conn.writeMtx.Lock()
+	_, err := conn.Write(writeBytes)
+	conn.writeMtx.Unlock()
+	return err
+}
+
+func (conn *underlayTCP) WriteBuffer(data []byte) error {
+	conn.writeMtx.Lock()
+	_, err := conn.Write(data)
+	conn.writeMtx.Unlock()
+	return err
+}
+
+func (conn *underlayTCP) WriteMessage(mainType uint16, subType uint16, packet interface{}) error {
+	// TODO: call newMessage
+	data, err := json.Marshal(packet)
+	if err != nil {
+		return err
+	}
+	writeBytes := append(encodeHeader(mainType, subType, uint32(len(data))), data...)
+	conn.writeMtx.Lock()
+	_, err = conn.Write(writeBytes)
+	conn.writeMtx.Unlock()
+	return err
+}
+
+func (conn *underlayTCP) Close() error {
+	return conn.Conn.Close()
+}
+
+func listenTCP(host string, port int, localPort int, mode string) (*underlayTCP, error) {
+	if mode == LinkModeTCPPunch {
+		c, err := reuse.DialTimeout("tcp", fmt.Sprintf("0.0.0.0:%d", localPort), fmt.Sprintf("%s:%d", host, port), SymmetricHandshakeAckTimeout) // TODO: timeout
+		if err != nil {
+			gLog.Println(LvDEBUG, "send tcp punch: ", err)
+			return nil, err
+		}
+		return &underlayTCP{writeMtx: &sync.Mutex{}, Conn: c}, nil
+	}
+	addr, _ := net.ResolveTCPAddr("tcp", fmt.Sprintf("0.0.0.0:%d", localPort))
+	l, err := net.ListenTCP("tcp", addr)
+	if err != nil {
+		return nil, err
+	}
+	l.SetDeadline(time.Now().Add(SymmetricHandshakeAckTimeout))
+	c, err := l.Accept()
+	defer l.Close()
+	if err != nil {
+		return nil, err
+	}
+	return &underlayTCP{writeMtx: &sync.Mutex{}, Conn: c}, nil
+}
+
+func dialTCP(host string, port int, localPort int, mode string) (*underlayTCP, error) {
+	var c net.Conn
+	var err error
+	if mode == LinkModeTCPPunch {
+		c, err = reuse.DialTimeout("tcp", fmt.Sprintf("0.0.0.0:%d", localPort), fmt.Sprintf("%s:%d", host, port), SymmetricHandshakeAckTimeout)
+	} else {
+		c, err = net.DialTimeout("tcp", fmt.Sprintf("%s:%d", host, port), SymmetricHandshakeAckTimeout)
+	}
+
+	if err != nil {
+		gLog.Printf(LvERROR, "Dial %s:%d error:%s", host, port, err)
+		return nil, err
+	}
+	gLog.Printf(LvDEBUG, "Dial %s:%d OK", host, port)
+	return &underlayTCP{writeMtx: &sync.Mutex{}, Conn: c}, nil
+}

underlay_tcp6.go

@@ -0,0 +1,92 @@
+package main
+
+import (
+	"encoding/json"
+	"fmt"
+	"io"
+	"net"
+	"sync"
+	"time"
+)
+
+type underlayTCP6 struct {
+	listener net.Listener
+	writeMtx *sync.Mutex
+	net.Conn
+}
+
+func (conn *underlayTCP6) Protocol() string {
+	return "tcp6"
+}
+
+func (conn *underlayTCP6) ReadBuffer() (*openP2PHeader, []byte, error) {
+	headBuf := make([]byte, openP2PHeaderSize)
+	_, err := io.ReadFull(conn, headBuf)
+	if err != nil {
+		return nil, nil, err
+	}
+	head, err := decodeHeader(headBuf)
+	if err != nil {
+		return nil, nil, err
+	}
+	dataBuf := make([]byte, head.DataLen)
+	_, err = io.ReadFull(conn, dataBuf)
+	return head, dataBuf, err
+}
+
+func (conn *underlayTCP6) WriteBytes(mainType uint16, subType uint16, data []byte) error {
+	writeBytes := append(encodeHeader(mainType, subType, uint32(len(data))), data...)
+	conn.writeMtx.Lock()
+	_, err := conn.Write(writeBytes)
+	conn.writeMtx.Unlock()
+	return err
+}
+
+func (conn *underlayTCP6) WriteBuffer(data []byte) error {
+	conn.writeMtx.Lock()
+	_, err := conn.Write(data)
+	conn.writeMtx.Unlock()
+	return err
+}
+
+func (conn *underlayTCP6) WriteMessage(mainType uint16, subType uint16, packet interface{}) error {
+	// TODO: call newMessage
+	data, err := json.Marshal(packet)
+	if err != nil {
+		return err
+	}
+	writeBytes := append(encodeHeader(mainType, subType, uint32(len(data))), data...)
+	conn.writeMtx.Lock()
+	_, err = conn.Write(writeBytes)
+	conn.writeMtx.Unlock()
+	return err
+}
+
+func (conn *underlayTCP6) Close() error {
+	return conn.Conn.Close()
+}
+
+func listenTCP6(port int, idleTimeout time.Duration) (*underlayTCP6, error) {
+	addr, _ := net.ResolveTCPAddr("tcp6", fmt.Sprintf("[::]:%d", port))
+	l, err := net.ListenTCP("tcp6", addr)
+	if err != nil {
+		return nil, err
+	}
+	defer l.Close()
+	l.SetDeadline(time.Now().Add(SymmetricHandshakeAckTimeout))
+	c, err := l.Accept()
+	defer l.Close()
+	if err != nil {
+		return nil, err
+	}
+	return &underlayTCP6{writeMtx: &sync.Mutex{}, Conn: c}, nil
+}
+
+func dialTCP6(host string, port int) (*underlayTCP6, error) {
+	c, err := net.DialTimeout("tcp6", fmt.Sprintf("[%s]:%d", host, port), SymmetricHandshakeAckTimeout)
+	if err != nil {
+		gLog.Printf(LvERROR, "Dial %s:%d error:%s", host, port, err)
+		return nil, err
+	}
+	return &underlayTCP6{writeMtx: &sync.Mutex{}, Conn: c}, nil
+}

update.go

@@ -16,9 +16,9 @@ import (
 	"time"
 )
 
-func update() {
-	gLog.Println(LevelINFO, "update start")
-	defer gLog.Println(LevelINFO, "update end")
+func update(host string, port int) {
+	gLog.Println(LvINFO, "update start")
+	defer gLog.Println(LvINFO, "update end")
 	c := http.Client{
 		Transport: &http.Transport{
 			TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
@@ -27,45 +27,45 @@ func update() {
 	}
 	goos := runtime.GOOS
 	goarch := runtime.GOARCH
-	rsp, err := c.Get(fmt.Sprintf("https://openp2p.cn:27183/api/v1/update?fromver=%s&os=%s&arch=%s", OpenP2PVersion, goos, goarch))
+	rsp, err := c.Get(fmt.Sprintf("https://%s:%d/api/v1/update?fromver=%s&os=%s&arch=%s", host, port, OpenP2PVersion, goos, goarch))
 	if err != nil {
-		gLog.Println(LevelERROR, "update:query update list failed:", err)
+		gLog.Println(LvERROR, "update:query update list failed:", err)
 		return
 	}
 	defer rsp.Body.Close()
 	if rsp.StatusCode != http.StatusOK {
-		gLog.Println(LevelERROR, "get update info error:", rsp.Status)
+		gLog.Println(LvERROR, "get update info error:", rsp.Status)
 		return
 	}
 	rspBuf, err := ioutil.ReadAll(rsp.Body)
 	if err != nil {
-		gLog.Println(LevelERROR, "update:read update list failed:", err)
+		gLog.Println(LvERROR, "update:read update list failed:", err)
 		return
 	}
 	updateInfo := UpdateInfo{}
 	err = json.Unmarshal(rspBuf, &updateInfo)
 	if err != nil {
-		gLog.Println(LevelERROR, rspBuf, " update info decode error:", err)
+		gLog.Println(LvERROR, rspBuf, " update info decode error:", err)
 		return
 	}
 	if updateInfo.Error != 0 {
-		gLog.Println(LevelERROR, "update error:", updateInfo.Error, updateInfo.ErrorDetail)
+		gLog.Println(LvERROR, "update error:", updateInfo.Error, updateInfo.ErrorDetail)
 		return
 	}
 	err = updateFile(updateInfo.Url, "", "openp2p")
 	if err != nil {
-		gLog.Println(LevelERROR, "update: download failed:", err)
+		gLog.Println(LvERROR, "update: download failed:", err)
 		return
 	}
 }
 
 // todo rollback on error
 func updateFile(url string, checksum string, dst string) error {
-	gLog.Println(LevelINFO, "download ", url)
+	gLog.Println(LvINFO, "download ", url)
 	tmpFile := filepath.Dir(os.Args[0]) + "/openp2p.tmp"
 	output, err := os.OpenFile(tmpFile, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0776)
 	if err != nil {
-		gLog.Printf(LevelERROR, "OpenFile %s error:%s", tmpFile, err)
+		gLog.Printf(LvERROR, "OpenFile %s error:%s", tmpFile, err)
 		return err
 	}
 	tr := &http.Transport{
@@ -74,31 +74,31 @@ func updateFile(url string, checksum string, dst string) error {
 	client := &http.Client{Transport: tr}
 	response, err := client.Get(url)
 	if err != nil {
-		gLog.Printf(LevelERROR, "download url %s error:%s", url, err)
+		gLog.Printf(LvERROR, "download url %s error:%s", url, err)
 		output.Close()
 		return err
 	}
 	defer response.Body.Close()
 	n, err := io.Copy(output, response.Body)
 	if err != nil {
-		gLog.Printf(LevelERROR, "io.Copy error:%s", err)
+		gLog.Printf(LvERROR, "io.Copy error:%s", err)
 		output.Close()
 		return err
 	}
 	output.Sync()
 	output.Close()
-	gLog.Println(LevelINFO, "download ", url, " ok")
-	gLog.Printf(LevelINFO, "size: %d bytes", n)
+	gLog.Println(LvINFO, "download ", url, " ok")
+	gLog.Printf(LvINFO, "size: %d bytes", n)
 
 	err = os.Rename(os.Args[0], os.Args[0]+"0")
 	if err != nil && os.IsExist(err) {
-		gLog.Printf(LevelINFO, " rename %s error:%s", os.Args[0], err)
+		gLog.Printf(LvINFO, " rename %s error:%s", os.Args[0], err)
 	}
 	// extract
-	gLog.Println(LevelINFO, "extract files")
+	gLog.Println(LvINFO, "extract files")
 	err = extract(filepath.Dir(os.Args[0]), tmpFile)
 	if err != nil {
-		gLog.Printf(LevelERROR, "extract error:%s. revert rename", err)
+		gLog.Printf(LvERROR, "extract error:%s. revert rename", err)
 		os.Rename(os.Args[0]+"0", os.Args[0])
 		return err
 	}

upnp.go

@@ -0,0 +1,385 @@
+/*
+Taken from taipei-torrent And fix some bugs
+*/
+package main
+
+import (
+	"bytes"
+	"encoding/xml"
+	"errors"
+	"fmt"
+	"io/ioutil"
+	"net"
+	"net/http"
+	"strconv"
+	"strings"
+	"time"
+)
+
+type upnpNAT struct {
+	serviceURL string
+	ourIP      string
+	urnDomain  string
+}
+
+// protocol is either "udp" or "tcp"
+type NAT interface {
+	GetExternalAddress() (addr net.IP, err error)
+	AddPortMapping(protocol string, externalPort, internalPort int, description string, timeout int) (mappedExternalPort int, err error)
+	DeletePortMapping(protocol string, externalPort, internalPort int) (err error)
+}
+
+func Discover() (nat NAT, err error) {
+	localIP := localIPv4()
+	ssdp, err := net.ResolveUDPAddr("udp4", "239.255.255.250:1900")
+	if err != nil {
+		return
+	}
+	conn, err := net.ListenPacket("udp4", fmt.Sprintf("%s:0", localIP))
+	if err != nil {
+		return
+	}
+	socket := conn.(*net.UDPConn)
+	defer socket.Close() // nolint: errcheck
+
+	if err := socket.SetDeadline(time.Now().Add(3 * time.Second)); err != nil {
+		return nil, err
+	}
+
+	st := "InternetGatewayDevice:1"
+
+	buf := bytes.NewBufferString(
+		"M-SEARCH * HTTP/1.1\r\n" +
+			"HOST: 239.255.255.250:1900\r\n" +
+			"ST: ssdp:all\r\n" +
+			"MAN: \"ssdp:discover\"\r\n" +
+			"MX: 2\r\n\r\n")
+	message := buf.Bytes()
+	answerBytes := make([]byte, 1024)
+	for i := 0; i < 3; i++ {
+		_, err = socket.WriteToUDP(message, ssdp)
+		if err != nil {
+			return
+		}
+		var n int
+		_, _, err = socket.ReadFromUDP(answerBytes)
+		if err != nil {
+			gLog.Println(LvDEBUG, "UPNP discover error:", err)
+			return
+		}
+
+		for {
+			n, _, err = socket.ReadFromUDP(answerBytes)
+			if err != nil {
+				break
+			}
+			answer := string(answerBytes[0:n])
+			if !strings.Contains(answer, st) {
+				continue
+			}
+			// HTTP header field names are case-insensitive.
+			// http://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2
+			locString := "\r\nlocation:"
+			answer = strings.ToLower(answer)
+			locIndex := strings.Index(answer, locString)
+			if locIndex < 0 {
+				continue
+			}
+			loc := answer[locIndex+len(locString):]
+			endIndex := strings.Index(loc, "\r\n")
+			if endIndex < 0 {
+				continue
+			}
+			locURL := strings.TrimSpace(loc[0:endIndex])
+			var serviceURL, urnDomain string
+			serviceURL, urnDomain, err = getServiceURL(locURL)
+			if err != nil {
+				return
+			}
+			if err != nil {
+				return
+			}
+			nat = &upnpNAT{serviceURL: serviceURL, ourIP: localIP, urnDomain: urnDomain}
+			return
+		}
+	}
+	err = errors.New("UPnP port discovery failed")
+	return
+}
+
+type Envelope struct {
+	XMLName xml.Name `xml:"http://schemas.xmlsoap.org/soap/envelope/ Envelope"`
+	Soap    *SoapBody
+}
+type SoapBody struct {
+	XMLName    xml.Name `xml:"http://schemas.xmlsoap.org/soap/envelope/ Body"`
+	ExternalIP *ExternalIPAddressResponse
+}
+
+type ExternalIPAddressResponse struct {
+	XMLName   xml.Name `xml:"GetExternalIPAddressResponse"`
+	IPAddress string   `xml:"NewExternalIPAddress"`
+}
+
+type ExternalIPAddress struct {
+	XMLName xml.Name `xml:"NewExternalIPAddress"`
+	IP      string
+}
+
+type UPNPService struct {
+	ServiceType string `xml:"serviceType"`
+	ControlURL  string `xml:"controlURL"`
+}
+
+type DeviceList struct {
+	Device []Device `xml:"device"`
+}
+
+type ServiceList struct {
+	Service []UPNPService `xml:"service"`
+}
+
+type Device struct {
+	XMLName     xml.Name    `xml:"device"`
+	DeviceType  string      `xml:"deviceType"`
+	DeviceList  DeviceList  `xml:"deviceList"`
+	ServiceList ServiceList `xml:"serviceList"`
+}
+
+type Root struct {
+	Device Device
+}
+
+func getChildDevice(d *Device, deviceType string) *Device {
+	dl := d.DeviceList.Device
+	for i := 0; i < len(dl); i++ {
+		if strings.Contains(dl[i].DeviceType, deviceType) {
+			return &dl[i]
+		}
+	}
+	return nil
+}
+
+func getChildService(d *Device, serviceType string) *UPNPService {
+	sl := d.ServiceList.Service
+	for i := 0; i < len(sl); i++ {
+		if strings.Contains(sl[i].ServiceType, serviceType) {
+			return &sl[i]
+		}
+	}
+	return nil
+}
+
+func localIPv4() string { // TODO: multi nic will wrong
+	conn, err := net.Dial("udp", "8.8.8.8:80")
+	if err != nil {
+		return ""
+	}
+	defer conn.Close()
+	localAddr := conn.LocalAddr().(*net.UDPAddr)
+	return localAddr.IP.String()
+}
+
+func getServiceURL(rootURL string) (url, urnDomain string, err error) {
+	r, err := http.Get(rootURL)
+	if err != nil {
+		return
+	}
+	defer r.Body.Close() // nolint: errcheck
+
+	if r.StatusCode >= 400 {
+		err = errors.New(fmt.Sprint(r.StatusCode))
+		return
+	}
+	var root Root
+	err = xml.NewDecoder(r.Body).Decode(&root)
+	if err != nil {
+		return
+	}
+	a := &root.Device
+	if !strings.Contains(a.DeviceType, "InternetGatewayDevice:1") {
+		err = errors.New("No InternetGatewayDevice")
+		return
+	}
+
+	b := getChildDevice(a, "WANDevice:1")
+	if b == nil {
+		err = errors.New("No WANDevice")
+		return
+	}
+	c := getChildDevice(b, "WANConnectionDevice:1")
+	if c == nil {
+		err = errors.New("No WANConnectionDevice")
+		return
+	}
+	d := getChildService(c, "WANIPConnection:1")
+	if d == nil {
+		// Some routers don't follow the UPnP spec, and put WanIPConnection under WanDevice,
+		// instead of under WanConnectionDevice
+		d = getChildService(b, "WANIPConnection:1")
+
+		if d == nil {
+			d = getChildService(c, "WANPPPConnection:1")
+			if d == nil {
+				err = errors.New("No WANIPConnection or WANPPPConnection")
+				return
+			}
+
+		}
+	}
+	// Extract the domain name, which isn't always 'schemas-upnp-org'
+	urnDomain = strings.Split(d.ServiceType, ":")[1]
+	url = combineURL(rootURL, d.ControlURL)
+	return
+}
+
+func combineURL(rootURL, subURL string) string {
+	protocolEnd := "://"
+	protoEndIndex := strings.Index(rootURL, protocolEnd)
+	a := rootURL[protoEndIndex+len(protocolEnd):]
+	rootIndex := strings.Index(a, "/")
+	return rootURL[0:protoEndIndex+len(protocolEnd)+rootIndex] + subURL
+}
+
+func soapRequest(url, function, message, domain string) (r *http.Response, err error) {
+	fullMessage := "<?xml version=\"1.0\" ?>" +
+		"<s:Envelope xmlns:s=\"http://schemas.xmlsoap.org/soap/envelope/\" s:encodingStyle=\"http://schemas.xmlsoap.org/soap/encoding/\">\r\n" +
+		"<s:Body>" + message + "</s:Body></s:Envelope>"
+
+	req, err := http.NewRequest("POST", url, strings.NewReader(fullMessage))
+	if err != nil {
+		return nil, err
+	}
+	req.Header.Set("Content-Type", "text/xml ; charset=\"utf-8\"")
+	req.Header.Set("User-Agent", "Darwin/10.0.0, UPnP/1.0, MiniUPnPc/1.3")
+	//req.Header.Set("Transfer-Encoding", "chunked")
+	req.Header.Set("SOAPAction", "\"urn:"+domain+":service:WANIPConnection:1#"+function+"\"")
+	req.Header.Set("Connection", "Close")
+	req.Header.Set("Cache-Control", "no-cache")
+	req.Header.Set("Pragma", "no-cache")
+
+	// log.Stderr("soapRequest ", req)
+
+	r, err = http.DefaultClient.Do(req)
+	if err != nil {
+		return nil, err
+	}
+	/*if r.Body != nil {
+	    defer r.Body.Close()
+	}*/
+
+	if r.StatusCode >= 400 {
+		// log.Stderr(function, r.StatusCode)
+		err = errors.New("Error " + strconv.Itoa(r.StatusCode) + " for " + function)
+		r = nil
+		return
+	}
+	return
+}
+
+type statusInfo struct {
+	externalIpAddress string
+}
+
+func (n *upnpNAT) getExternalIPAddress() (info statusInfo, err error) {
+
+	message := "<u:GetExternalIPAddress xmlns:u=\"urn:" + n.urnDomain + ":service:WANIPConnection:1\">\r\n" +
+		"</u:GetExternalIPAddress>"
+
+	var response *http.Response
+	response, err = soapRequest(n.serviceURL, "GetExternalIPAddress", message, n.urnDomain)
+	if response != nil {
+		defer response.Body.Close() // nolint: errcheck
+	}
+	if err != nil {
+		return
+	}
+
+	var envelope Envelope
+	data, err := ioutil.ReadAll(response.Body)
+	if err != nil {
+		return
+	}
+
+	reader := bytes.NewReader(data)
+	err = xml.NewDecoder(reader).Decode(&envelope)
+	if err != nil {
+		return
+	}
+
+	info = statusInfo{envelope.Soap.ExternalIP.IPAddress}
+
+	if err != nil {
+		return
+	}
+
+	return
+}
+
+// GetExternalAddress returns an external IP. If GetExternalIPAddress action
+// fails or IP returned is invalid, GetExternalAddress returns an error.
+func (n *upnpNAT) GetExternalAddress() (addr net.IP, err error) {
+	info, err := n.getExternalIPAddress()
+	if err != nil {
+		return
+	}
+	addr = net.ParseIP(info.externalIpAddress)
+	if addr == nil {
+		err = fmt.Errorf("Failed to parse IP: %v", info.externalIpAddress)
+	}
+
+	return
+}
+
+func (n *upnpNAT) AddPortMapping(protocol string, externalPort, internalPort int, description string, timeout int) (mappedExternalPort int, err error) {
+	// A single concatenation would break ARM compilation.
+	message := "<u:AddPortMapping xmlns:u=\"urn:" + n.urnDomain + ":service:WANIPConnection:1\">\r\n" +
+		"<NewRemoteHost></NewRemoteHost><NewExternalPort>" + strconv.Itoa(externalPort)
+	message += "</NewExternalPort><NewProtocol>" + protocol + "</NewProtocol>"
+	message += "<NewInternalPort>" + strconv.Itoa(internalPort) + "</NewInternalPort>" +
+		"<NewInternalClient>" + n.ourIP + "</NewInternalClient>" +
+		"<NewEnabled>1</NewEnabled><NewPortMappingDescription>"
+	message += description +
+		"</NewPortMappingDescription><NewLeaseDuration>" + strconv.Itoa(timeout) +
+		"</NewLeaseDuration></u:AddPortMapping>"
+
+	var response *http.Response
+	response, err = soapRequest(n.serviceURL, "AddPortMapping", message, n.urnDomain)
+	if response != nil {
+		defer response.Body.Close() // nolint: errcheck
+	}
+	if err != nil {
+		return
+	}
+
+	// TODO: check response to see if the port was forwarded
+	// log.Println(message, response)
+	// JAE:
+	// body, err := ioutil.ReadAll(response.Body)
+	// fmt.Println(string(body), err)
+	mappedExternalPort = externalPort
+	_ = response
+	return
+}
+
+func (n *upnpNAT) DeletePortMapping(protocol string, externalPort, internalPort int) (err error) {
+
+	message := "<u:DeletePortMapping xmlns:u=\"urn:" + n.urnDomain + ":service:WANIPConnection:1\">\r\n" +
+		"<NewRemoteHost></NewRemoteHost><NewExternalPort>" + strconv.Itoa(externalPort) +
+		"</NewExternalPort><NewProtocol>" + protocol + "</NewProtocol>" +
+		"</u:DeletePortMapping>"
+
+	var response *http.Response
+	response, err = soapRequest(n.serviceURL, "DeletePortMapping", message, n.urnDomain)
+	if response != nil {
+		defer response.Body.Close() // nolint: errcheck
+	}
+	if err != nil {
+		return
+	}
+
+	// TODO: check response to see if the port was deleted
+	// log.Println(message, response)
+	_ = response
+	return
+}

util_darwin.go

@@ -1,6 +1,3 @@
-//go:build darwin
-// +build darwin
-
 package main
 
 import (

util_linux.go

@@ -1,6 +1,3 @@
-//go:build linux
-// +build linux
-
 package main
 
 import (

util_windows.go

@@ -1,6 +1,3 @@
-//go:build windows
-// +build windows
-
 package main
 
 import (
@@ -38,7 +35,7 @@ func setRLimit() error {
 func setFirewall() {
 	fullPath, err := filepath.Abs(os.Args[0])
 	if err != nil {
-		gLog.Println(LevelERROR, "add firewall error:", err)
+		gLog.Println(LvERROR, "add firewall error:", err)
 		return
 	}
 	isXP := false