Skip to content

feat(client): add usage reporting #2369

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 62 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
58484de
Pass in serverName to VpnTunnelService
62w71st Jul 16, 2024
304748c
Pass in serverName to VpnTunnelService
62w71st Jul 16, 2024
ec80311
merge
62w71st Jul 20, 2024
bd4c0f9
Merge branch 'master' of https://github.com/62w71st/outline-apps
62w71st Jul 20, 2024
b5ce091
merge
62w71st Jul 24, 2024
2716f8b
add servername to cordova and electron
62w71st Jul 25, 2024
85e688f
change the origin of the outline plugin code
Jul 26, 2024
321868f
conditionally pass serverName for Cordova only on Android.
Jul 26, 2024
e44ced5
lint
Jul 26, 2024
3ad060e
merge with upstream
62w71st Jul 31, 2024
0ea80b0
resolve conflicts
62w71st Jul 31, 2024
337695c
conflict resolution part 2.
62w71st Jul 31, 2024
b4e2c51
fix compilation errors
Jul 31, 2024
43a9b80
Merge branch 'master' of https://github.com/62w71st/outline-apps
Jul 31, 2024
ecaedf2
Add serverName to logs and remove getServerName
Aug 5, 2024
eb0316c
remove ongoing
Aug 5, 2024
fb2080e
Merge branch 'master' into master
62w71st Aug 5, 2024
5b9acb1
Merge branch 'master' of https://github.com/Jigsaw-Code/outline-apps
62w71st Aug 8, 2024
225b9da
Add reporting client.
62w71st Jan 31, 2025
8ba5d02
usage-reporting parser.
62w71st Feb 6, 2025
ca66235
Merge remote-tracking branch 'upstream/master' into usage-reporting-v2
62w71st Feb 7, 2025
fc21f96
Merge branch 'Jigsaw-Code:master' into usage-reporting-v2
62w71st Feb 17, 2025
21dd88f
Merge branch 'master' of https://github.com/Jigsaw-Code/outline-apps …
62w71st Feb 17, 2025
2e46848
Merge branch 'usage-reporting-v2' of https://github.com/62w71st/outli…
62w71st Feb 17, 2025
56d5cfe
GET -> POST
62w71st Feb 17, 2025
3aca920
Merge branch 'master' of https://github.com/Jigsaw-Code/outline-apps …
62w71st Feb 22, 2025
3e1da1f
Add OnConnected() callback.
62w71st Feb 22, 2025
6718e50
fix build
62w71st Feb 28, 2025
72a72ff
Add connect and disconnect to the outline client
62w71st Feb 28, 2025
981b656
fix build errors
62w71st Feb 28, 2025
e6a06fe
Merge branch 'master' of https://github.com/Jigsaw-Code/outline-apps …
62w71st Mar 7, 2025
23d8230
pass client as streamDialer to reporting client
62w71st Mar 10, 2025
65a7245
Merge remote-tracking branch 'upstream/master' into usage-reporting-v2
62w71st Mar 15, 2025
defff32
add parsing 1/2
62w71st Mar 21, 2025
0ad54a5
create usagereporter in the client
62w71st Mar 23, 2025
2873ce6
merge changes from upstream master
62w71st Mar 28, 2025
32fe879
add session client
62w71st Apr 11, 2025
d747c6a
Fix
62w71st Apr 11, 2025
303a555
fix build errors in OutlineAndroidLib
62w71st Apr 11, 2025
14e3719
Add usage report config to ts and java files
62w71st Apr 11, 2025
812c32e
Separate the session client from the transport client.
62w71st Apr 18, 2025
e7a7784
fix build error.
62w71st Apr 18, 2025
63061e7
export getters in sessionclient for gobind to export SessionClient!
62w71st Apr 18, 2025
f8aaa80
Fix build.
62w71st Apr 18, 2025
42a20de
Fix Getter.
62w71st Apr 18, 2025
341455e
gobind
62w71st Apr 18, 2025
5c1185b
Merge branch 'Jigsaw-Code:master' into usage-reporting-v2
62w71st Apr 27, 2025
a1c3bed
Addressed a few comments and removed usage reporting configuration fr…
62w71st Apr 27, 2025
ed0d9d4
Removed usage reporting config from TunnelConfig.aidl
62w71st Apr 27, 2025
78449ea
Merge branch 'master' of https://github.com/Jigsaw-Code/outline-apps …
62w71st May 3, 2025
e6f0175
remove cookies.json
62w71st May 4, 2025
0bf84ae
merge upstream
62w71st May 9, 2025
edc200a
Fix tests.
62w71st May 9, 2025
5d9a16c
Pass in session_report to the cordova plugin.
62w71st May 9, 2025
b7f529c
Fix module_test.go tests.
62w71st May 15, 2025
c7dbf80
stop reporting when tunnel is stopped.
62w71st May 15, 2025
b14c48a
Merge branch 'master' into usage-reporting-v2
62w71st May 15, 2025
896c7b9
Merge branch 'Jigsaw-Code:master' into usage-reporting-v2
62w71st May 26, 2025
b33ec07
Apply comments.
62w71st May 26, 2025
557972b
Merge branch 'usage-reporting-v2' of https://github.com/62w71st/outli…
62w71st May 26, 2025
ecdcef1
Apply comments.
62w71st May 26, 2025
4e75174
Merge remote-tracking branch 'upstream/master' into usage-reporting-v2
62w71st Jun 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions client/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,27 @@ Note that Websockets is not yet supported on Windows. In order to have a single

```yaml
transport:
$type: first-supported
options:
- $type: tcpudp
tcp: &shared
$type: shadowsocks
endpoint: example.com:80
cipher: chacha20-ietf-poly1305
secret: SECRET

udp: *shared

session_report:
$type: sessionreport
url: https://your-callback-server.com/outline_callback
interval: 24h
enable_cookies: true`)

tcpudp with session
- $type: tcpudp without session
<<: &tcpudp

$type: tcpudp
tcp:
$type: shadowsocks
Expand Down
61 changes: 54 additions & 7 deletions client/go/outline/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// distributed under the License is distributed on an "AS IS BASIS,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Plz revert

Suggested change
// distributed under the License is distributed on an "AS IS BASIS,
// distributed under the License is distributed on an "AS IS" BASIS,

// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
Expand All @@ -17,10 +17,12 @@ package outline
import (
"context"
"errors"
"fmt"
"net"

"github.com/Jigsaw-Code/outline-apps/client/go/outline/config"
"github.com/Jigsaw-Code/outline-apps/client/go/outline/platerrors"
"github.com/Jigsaw-Code/outline-apps/client/go/outline/reporting"
"github.com/Jigsaw-Code/outline-sdk/transport"
"github.com/goccy/go-yaml"
)
Expand All @@ -30,8 +32,10 @@ import (
// It's used by the connectivity test and the tun2socks handlers.
// TODO: Rename to Transport. Needs to update per-platform code.
type Client struct {
sd *config.Dialer[transport.StreamConn]
pl *config.PacketListener
sd *config.Dialer[transport.StreamConn]
pl *config.PacketListener
ur *config.UsageReporter
cancel context.CancelFunc // Used to stop reporting
}

func (c *Client) DialStream(ctx context.Context, address string) (transport.StreamConn, error) {
Expand All @@ -55,18 +59,35 @@ type NewClientResult struct {
Error *platerrors.PlatformError
}

func (c *Client) StartReporting() {
if c.ur == nil {
return
}
ctx, cancel := context.WithCancel(context.Background())
c.cancel = cancel // Store the cancel function to stop reporting later

go reporting.StartReporting(ctx, c, c.ur)
}

func (c *Client) StopReporting() {
if c.cancel != nil {
c.cancel() // Signal the context to stop reporting
c.cancel = nil
}
}

// NewClient creates a new Outline client from a configuration string.
func NewClient(clientConfig string) *NewClientResult {
func NewClient(clientConfig string, sessionConfig string) *NewClientResult {
tcpDialer := transport.TCPDialer{Dialer: net.Dialer{KeepAlive: -1}}
udpDialer := transport.UDPDialer{}
client, err := NewClientWithBaseDialers(clientConfig, &tcpDialer, &udpDialer)
client, err := NewClientWithBaseDialers(clientConfig, sessionConfig, &tcpDialer, &udpDialer)
if err != nil {
return &NewClientResult{Error: platerrors.ToPlatformError(err)}
}
return &NewClientResult{Client: client}
}

func NewClientWithBaseDialers(clientConfigText string, tcpDialer transport.StreamDialer, udpDialer transport.PacketDialer) (*Client, error) {
func NewClientWithBaseDialers(clientConfigText string, sessionConfig string, tcpDialer transport.StreamDialer, udpDialer transport.PacketDialer) (*Client, error) {
var clientConfig ClientConfig
err := yaml.Unmarshal([]byte(clientConfigText), &clientConfig)
if err != nil {
Expand Down Expand Up @@ -108,5 +129,31 @@ func NewClientWithBaseDialers(clientConfigText string, tcpDialer transport.Strea
}
}

return &Client{sd: transportPair.StreamDialer, pl: transportPair.PacketListener}, nil
usageReportYAML, err := config.ParseConfigYAML(sessionConfig)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this sessionConfig specific to sessionReportingConfig? Or can it be used for other types of sessionConfig in the future? If it's the latter, we should probably implement type assertion to ensure the type is 'reporting'.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the easiest is to move the parsing of the clientConfig to the config module, which already takes care of this. And it allows you to use first_supported

if err != nil {
return nil, &platerrors.PlatformError{
Code: platerrors.InvalidConfig,
Message: "client config is not valid YAML",
Cause: platerrors.ToPlatformError(err),
}
}
usageReporter, err := config.NewUsageReportProvider().Parse(context.Background(), usageReportYAML)
if err != nil {
if errors.Is(err, errors.ErrUnsupported) {
return nil, &platerrors.PlatformError{
Code: platerrors.InvalidConfig,
Message: "unsupported client config",
Cause: platerrors.ToPlatformError(err),
}
} else {
return nil, &platerrors.PlatformError{
Code: platerrors.InvalidConfig,
Message: "failed to create usage report",
Cause: platerrors.ToPlatformError(err),
}
}
}
fmt.Println("usageReporter", usageReporter)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this for debugging purpose or logging purpose?


return &Client{sd: transportPair.StreamDialer, pl: transportPair.PacketListener, ur: usageReporter}, nil
}
54 changes: 40 additions & 14 deletions client/go/outline/client_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ func Test_NewTransport_SS_URL(t *testing.T) {
config := "transport: ss://[email protected]:4321/"
firstHop := "example.com:4321"

result := NewClient(config)
result := NewClient(config, "")
require.Nil(t, result.Error, "Got %v", result.Error)
require.Equal(t, firstHop, result.Client.sd.FirstHop)
require.Equal(t, firstHop, result.Client.pl.FirstHop)
Expand All @@ -41,7 +41,7 @@ transport: {
}`
firstHop := "example.com:4321"

result := NewClient(config)
result := NewClient(config, "")
require.Nil(t, result.Error, "Got %v", result.Error)
require.Equal(t, firstHop, result.Client.sd.FirstHop)
require.Equal(t, firstHop, result.Client.pl.FirstHop)
Expand All @@ -58,7 +58,7 @@ transport: {
}`
firstHop := "example.com:4321"

result := NewClient(config)
result := NewClient(config, "")
require.Nil(t, result.Error, "Got %v", result.Error)
require.Equal(t, firstHop, result.Client.sd.FirstHop)
require.Equal(t, firstHop, result.Client.pl.FirstHop)
Expand All @@ -74,7 +74,7 @@ transport:
password: SECRET`
firstHop := "example.com:4321"

result := NewClient(config)
result := NewClient(config, "")
require.Nil(t, result.Error, "Got %v", result.Error)
require.Equal(t, firstHop, result.Client.sd.FirstHop)
require.Equal(t, firstHop, result.Client.pl.FirstHop)
Expand All @@ -90,7 +90,7 @@ transport:
secret: SECRET`
firstHop := "example.com:4321"

result := NewClient(config)
result := NewClient(config, "")
require.Nil(t, result.Error, "Got %v", result.Error)
require.Equal(t, firstHop, result.Client.sd.FirstHop)
require.Equal(t, firstHop, result.Client.pl.FirstHop)
Expand All @@ -107,7 +107,7 @@ transport:
secret: SECRET`
firstHop := "entry.example.com:4321"

result := NewClient(config)
result := NewClient(config, "")
require.Nil(t, result.Error, "Got %v", result.Error)
require.Equal(t, firstHop, result.Client.sd.FirstHop)
require.Equal(t, firstHop, result.Client.pl.FirstHop)
Expand All @@ -128,7 +128,7 @@ transport:
secret: EXIT_SECRET`
firstHop := "entry.example.com:4321"

result := NewClient(config)
result := NewClient(config, "")
require.Nil(t, result.Error, "Got %v", result.Error)
require.Equal(t, firstHop, result.Client.sd.FirstHop)
require.Equal(t, firstHop, result.Client.pl.FirstHop)
Expand All @@ -150,12 +150,38 @@ transport:
cipher: chacha20-ietf-poly1305
secret: SECRET`

result := NewClient(config)
result := NewClient(config, "")
require.Nil(t, result.Error, "Got %v", result.Error)
require.Equal(t, "example.com:80", result.Client.sd.FirstHop)
require.Equal(t, "example.com:53", result.Client.pl.FirstHop)
}

func Test_SessionReport(t *testing.T) {
transportConfig := `
$type: tcpudp
tcp:
$type: shadowsocks
endpoint: example.com:80
cipher: chacha20-ietf-poly1305
secret: SECRET
prefix: "POST "
udp:
$type: shadowsocks
endpoint: example.com:53
cipher: chacha20-ietf-poly1305
secret: SECRET`
sessionConfig := `
$type: sessionreport
url: https://your-callback-server.com/outline_callback
interval: 24h
enable_cookies: true`

client := NewClient(transportConfig, sessionConfig)
require.Nil(t, client.Error, "Got %v", client.Error)
require.NotNil(t, client.Client.ur, "UsageReporter is nil")
require.Equal(t, "https://your-callback-server.com/outline_callback", client.Client.ur.Url)
}

func Test_NewTransport_YAML_Reuse(t *testing.T) {
config := `
transport:
Expand All @@ -170,7 +196,7 @@ transport:
prefix: "POST "`
firstHop := "example.com:4321"

result := NewClient(config)
result := NewClient(config, "")
require.Nil(t, result.Error, "Got %v", result.Error)
require.Equal(t, firstHop, result.Client.sd.FirstHop)
require.Equal(t, firstHop, result.Client.pl.FirstHop)
Expand All @@ -192,15 +218,15 @@ transport:
endpoint: example.com:53
<<: *cipher`

result := NewClient(config)
result := NewClient(config, "")
require.Nil(t, result.Error, "Got %v", result.Error)
require.Equal(t, "example.com:80", result.Client.sd.FirstHop)
require.Equal(t, "example.com:53", result.Client.pl.FirstHop)
}

func Test_NewTransport_Unsupported(t *testing.T) {
config := `transport: {$type: unsupported}`
result := NewClient(config)
result := NewClient(config, "")
require.Error(t, result.Error, "Got %v", result.Error)
require.Equal(t, "unsupported config", result.Error.Message)
}
Expand All @@ -223,7 +249,7 @@ transport:
url: https://entrypoint.cdn.example.com/udp`
firstHop := "entrypoint.cdn.example.com:443"

result := NewClient(config)
result := NewClient(config, "")
require.Nil(t, result.Error, "Got %v", result.Error)
require.Equal(t, firstHop, result.Client.sd.FirstHop)
require.Equal(t, firstHop, result.Client.pl.FirstHop)
Expand All @@ -235,7 +261,7 @@ transport:
$type: tcpudp
tcp:
udp:`
result := NewClient(config)
result := NewClient(config, "")
require.Error(t, result.Error, "Got %v", result.Error)
perr := &platerrors.PlatformError{}
require.ErrorAs(t, result.Error, &perr)
Expand Down Expand Up @@ -295,7 +321,7 @@ func Test_NewClientFromJSON_Errors(t *testing.T) {
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
got := NewClient(tt.input)
got := NewClient(tt.input, "")
if got.Error == nil || got.Client != nil {
t.Errorf("NewClientFromJSON() expects an error, got = %v", got.Client)
return
Expand Down
52 changes: 52 additions & 0 deletions client/go/outline/config/config_usage_reporter.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
// Copyright 2025 The Outline Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package config

import (
"context"
"fmt"
"time"
)

type UsageReporter struct {
Interval time.Duration
Url string
EnableCookies bool
}

// UsageReporterConfig is the format for the Usage Reporter config.
type UsageReporterConfig struct {
Interval string
Url string
EnableCookies bool `json:"enable_cookies"`
}

func parseUsageReporterConfig(ctx context.Context, configMap map[string]any) (*UsageReporter, error) {
var config UsageReporterConfig
if err := mapToAny(configMap, &config); err != nil {
return nil, fmt.Errorf("invalid config format: %w", err)
}

duration, err := time.ParseDuration(config.Interval)
if err != nil {
return nil, fmt.Errorf("failed to parse interval: %w", err)
}

return &UsageReporter{
Interval: duration,
Url: config.Url,
EnableCookies: config.EnableCookies,
}, nil
}
16 changes: 16 additions & 0 deletions client/go/outline/config/module.go
Original file line number Diff line number Diff line change
Expand Up @@ -142,3 +142,19 @@ func NewDefaultTransportProvider(tcpDialer transport.StreamDialer, udpDialer tra

return transports
}

func NewUsageReportProvider() *TypeParser[*UsageReporter] {
usageReporting := NewTypeParser(func(ctx context.Context, input ConfigNode) (*UsageReporter, error) {
switch input.(type) {
// An absent config is acceptable.
case nil:
return nil, nil
default:
return nil, errors.New("parser not specified")
}
Comment on lines +148 to +154
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we simplify return nil, errors.New("xxxxx") and guard the nil case in outline/cient.go? Will it cause any problems?

})

usageReporting.RegisterSubParser("sessionreport", parseUsageReporterConfig)

return usageReporting
}
Loading