We need to log all requests and response to the 3rd party services #

We want to store http request and response headers and data in log_logs table

Idea #

Variety of requests and responses #

• Some services providing direct access to http.Request object
• Some services providing an sdk without direct access to http.Request
• We also want to log incoming request / webhooks

Georgy found a great idea on how to use http.Transport field:

The mechanism #

Go’s http.Client executes HTTP requests via its Transport field, which implements http.RoundTripper. RoundTripper has one method: RoundTrip(*http.Request) (*http.Response, error). This is the heart of http.Client.

We need to extend http.Transport which implements RoundTripper and wraps other RoundTripper (normally http.DefaultTransport) with logging functionality.

Proposal one #

Turn on / off logging directly in app layer

app/app.go:

func (app *App) CreateTrade(ctx context.Context, pg PostgresRepository, profileID int, result interface{}) error {
  // some code

  // turn on logging when needed
	app.TurnOnAppLogging(   // pass all required paramters to logging
    ctx,
    app.repo,
  )

  app.dwolla.CreateTrade(ctx, ...)
  app.dwolla.FundTransfer(ctx, ...)

  app.TurnOffAppLogging()  // revert back default transporter
	return nil
}

app/logging.go:

import (
	"net/http"

	"github.com/webdevelop-pro/go-logging/http2db"
)

func (app *App) TurnOnAppLogging(ctx context.Context, pg PostgresRepository) error {
  http.DefaultTransport = http2db.PostgresTransport(ctx, pg)
}

http2db.go:

// similar to 
// https://github.com/motemen/go-loghttp/blob/master/loghttp.go
// but with store to database 

Pros #

• adapters can turn on / off logging when
• no need to change anything in adapters
• no need to pass any additional paramters to adapters to logging (like database connection)

Cons #

• Need to make changes in app
• custom code required for incoming wehook requests

Proposal two #

Turn on / off logging directly in adapters layer

adapters/dwolla.go:

func (dwolla Dwolla) TurnOnAppLogging(client *http.Client) {
	d.client.HTTPClient = client
}

func (dwolla Dwolla) CreateTrade(ctx context.Context, pg PostgresRepository, profileID int, result interface{}) error {
	dwolla.TurnOnAppLogging(   // pass all required paramters to logging
    ctx,
    app.repo,
  )

  dwolla.client.CreateTrade(ctx, ...)

  // revert back default transporter
  app.TurnOffAppLogging()  

	return nil
}

Pros #

• adapters can turn on / off logging when needed

Cons #

• Need to make changes in adapters
• Adapters need to have connection to database
• custom code required for incoming wehook requests

Future ideas #

• Migrate to enterprice ready solution
• Collect logs from pods
• Store logs to different postgres database or in loki