Golang Workspace¶

Intro¶

In [1]:
// %env GOPATH=/root/go
// %env PATH=/root/go:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
!echo $GOPATH
!echo $PATH
!go version

/root/go
/root/go/bin:/usr/local/bin:/usr/local/sbin:/usr/sbin:/usr/bin:/sbin:/bin
go version go1.19.8 linux/amd64

Golang Kernel¶

In [2]:
// reference documentation
%help

GoNB Help Page¶

GoNB is a Go kernel that compiles and executes on-the-fly Go code.

When executing a cell, GoNB will save the cell contents (except non-Go commands see below) into a main.go file, compile and execute it.

It also saves any global declarations (imports, functions, types, variables, constants) and reuse them at the next cell execution -- so you can define a function in one cell, and reuse in the next one. Just the func main() is not reused.

A hello world example would look like:

func main() {
    fmt.Printf(`Hello world!\n`);
}

But to avoid having to type func main() all the time, you can use %% and everything after is wrapped inside a func main() { ... }. So our revised hello world looks like:

%%
fmt.Printf(`Hello world!\n`)

Init Functions -- func init()¶

Since there is always only one definition per function name, it's not possible for each cell to have its own init() function. Instead, GoNB converts any function named init_something() to init() before compiling and executing. This way each cell can create its own init_...() and have it called at every cell execution.

Special non-Go Commands¶

  • %% or %main: Marks the lines as follows to be wrapped in a func main() {...} during execution. A shortcut to quickly execute code. It also automatically includes flag.Parse() as the very first statement. Anything %% or %main are taken as arguments to be passed to the program -- it resets previous values given by %args.
  • %args: Sets arguments to be passed when executing the Go code. This allows one to use flags as a normal program. Notice that if a value after %% or %main is given, it will overwrite the values here.
  • %autoget and %noautoget: Default is %autoget, which automatically does go get for packages not yet available.
  • %cd [<directory>]: Change current directory of the Go kernel, and the directory from where the cells are executed. If no directory is given it reports the current directory.
  • %env VAR value: Sets the environment variable VAR to the given value. These variables will be available both for Go code and for shell scripts.
  • %goflags <values...>: Configures list of extra arguments to pass to go build when compiling the code for execution of a cell. If no values are given, it simply shows the current setting. To reset its value, use %goflags """. See example on how to use this in the tutorial.
  • %with_inputs: will prompt for inputs for the next shell command. Use this if the next shell command (!) you execute reads the stdin. Jupyter will require you to enter one last value after the shell script executes.
  • %with_password: will prompt for a password passed to the next shell command. Do this is if your next shell command requires a password.

Notice all these commands are executed before any Go code in the same cell.

Managing Memorized Definitions¶

  • %list (or %ls): Lists all memorized definitions (imports, constants, types, variables and functions) that are carried from one cell to another.
  • %remove <definitions> (or %rm <definitions>): Removes (forgets) given definition(s). Use as key the value(s) listed with %ls.
  • %reset [go.mod] clears all memorized definitions (imports, constants, types, functions, etc.) as well as re-initializes the go.mod file. If the optional go.mod parameter is given, it will re-initialize only the go.mod file -- useful when testing different set up of versions of libraries.

Executing Shell Commands¶

  • !<shell_cmd>: executes the given command on a new shell. It makes it easy to run commands on the kernels box, for instance to install requirements, or quickly check contents of directories or files. Lines ending in \ are continued on the next line -- so multi-line commands can be entered. But each command is executed in its own shell, that is, variables and state is not carried over.
  • !*<shell_cmd>: same as !<shell_cmd> except it first changes directory to the temporary directory used to compile the go code -- the latest execution is always saved in the file main.go. It's also where the go.mod file for the notebook is created and maintained. Useful for manipulating go.mod, for instance to get a package from some specific version, something like !*go get github.com/my/package@v3.

Notice that when the cell is executed, first all shell commands are executed, and only after that, if there is any Go code in the cell, it is executed.

Tracking of Go Files In Development:¶

A convenient way to develop programs or libraries in GoNB is to use replace rules in GoNB's go.mod to your program or library being developed and test your program from GoNB -- see the Tutorial)'s section "Developing Go libraries with a notebook" for different ways of achieving this.

To manipulate the list of files tracked for changes:

  • %track [file_or_directory]: add file or directory to list of tracked files, which are monitored by GoNB (and 'gopls') for auto-complete or contextual help. If no file is given, it lists the currently tracked files.
  • %untrack [file_or_directory][...]: remove file or directory from list of tracked files. If suffixed with ... it will remove all files prefixed with the string given (without the ...). If no file is given, it lists the currently tracked files.

Environment Variables¶

For convenience, GoNB defines the following environment variables -- available for the shell scripts (! and !*) and for the Go cells:

  • GONB_DIR: the directory where commands are executed from. This can be changed with %cd.
  • GONB_TMP_DIR: the directory where the temporary Go code, with the cell code, is stored and compiled. This is the directory where !* scripts are executed. It only changes when a kernel is restarted, and a new temporary directory is created.
  • GONB_PIPE: is the named pipe directory used to communicate rich content (HTML, images) to the kernel. Only available for Go cells, and a new one is created at every execution. This is used by the `GoNBui`` functions described above, and doesn't need to be accessed directly.

Widgets¶

The package gonbui/widgets offers widgets that can be used to interact in a more dynamic way, using the HTML element in the browser. E.g.: buttons, sliders.

It's not necessary to do anything, but, to help debug the communication system with the front-end, GoNB offers a couple of special commands:

  • %widgets - install the javascript needed to communicate with the frontend. This is usually not needed, since it happens automatically when using Widgets.
  • %widgets_hb - send a heartbeat signal to the front-end and wait for the reply. Used for debugging only.

Writing for WASM (WebAssembly) (Experimental)¶

GoNB can also compile to WASM and run in the notebook. This is experimental, and likely to change (feedback is very welcome), and can be used to write interactive widgets in Go, in the notebook.

When a cell with %wasm is executed, a temporary directory is created under the Jupyter root directory called jupyter_files/<kernel unique id>/ and the cell is compiled to a wasm file and put in that directory.

Then GONB outputs the javascript needed to run the compiled wam.

In the Go code, the following extra constants/variables are created in the global namespace, and can be used in your Go code:

  • GonbWasmDir, GonbWasmUrl: the directory and url (served by Jupyter) where the generated .wasm files are read. Potentially, the user can use it to serve other files. These are unique for the kernel, but shared among cells.
  • GonbWasmDivId: When a %wasm cell is executed, an empty <div id="<unique_id>"></div> is created with a unique id -- every cell will have a different one. This is where the Wasm code can dynamically create content.

The following environment variables are set when %wasm is created:

  • GONB_WASM_SUBDIR, GONB_WASM_URL: the directory and url (served by Jupyter) where the generated .wasm files are read. Potentially, the user can use it to serve other files. These environment variables are available for shell scripts (!... and !*... special commands) and non-wasm programs if they want to serve different files from there.

Writing Tests and Benchmarks¶

If a cell includes the %test command (anywhere in cell), it is compiled with go test (as opposed to go build). This can be very useful both to demonstrate tests, or simply help develop/debug them in a notebook.

If %test is given without any flags, it uses by default the flags -test.v (verbose) and -test.run defined with the list of the tests defined in the current cell. That is, it will run only the tests in the current cell. Also, if there are any benchmarks in the current cell, it appends the flag -test.bench=. and runs the benchmarks defined in the current cell.

Alternatively one can use %test <flags>, and the flags are passed to the binary compiled with go test. Remember that test flags require to be prefixed with test.. So for a verbose output, use %test -test.v. For benchmarks, run %test -test.bench=. -test.run=Benchmark.

See examples in the gotest.ipynb notebook here.

Cell Magic¶

The following are special commands that change how the cell is interpreted, so they are prefixed with %% (two '%' symbols). They try to follow IPython's Cell Magic.

They must always appear as the first line of the cell.

The contents in the cells are not assumed to be Go, so auto-complete and contextual help are disabled in those cells.

%%writefile¶

%%writefile [-a] <filePath>

Write contents of the cell (except the first line with the '%%writefile') to the given <filePath>. If -a is given it will append the cell contents to the file.

This can be handy if for instance the notebook needs to write a configuration file, or simply to dump the code inside the cell into some file.

File path passes through a tilde (~) expansion to the user's home directory, as well as environment variable substitution (e.g.: ${HOME} or $MY_DIR/a/b).

%%script, %%bash and %%sh¶

%%script <command>

Execute <command> and feed it (STDIN) with the contents of the cell. The %%bash and %%sh magic is an alias to %%script bash and %%script sh respectively.

Generally, a convenient way to run larger scripts.

Other¶

  • %goworkfix: work around 'go get' inability to handle 'go.work' files. If you are using 'go.work' file to point to locally modified modules, consider using this. It creates 'go mod edit --replace' rules to point to the modules pointed to the 'use' rules in 'go.work' file. It overwrites/updates 'replace' rules for those modules, if they already exist. See tutorial for an example.

Links¶

  • github.com/janpfeifer/gonb - GitHub page.
  • Tutorial.
  • go.dev package reference.
In [3]:
import "fmt"

%%
fmt.Println("Hello, Gianni!")

Hello, Gianni!

Useful and Versatile Go Code Snippets¶

by Beck Moulton published on Medium

Timing Execution¶

In [4]:
import "fmt"
// Utility

func TrackTime(pre time.Time) time.Duration {
 elapsed := time.Since(pre)
 fmt.Println("elapsed:", elapsed)
 return elapsed
}

%%
defer TrackTime(time.Now()) // <--- THIS
time.Sleep(500 * time.Millisecond)

elapsed: 500.758575ms

Two-Stage Deferred Execution¶

In [5]:
func setupTeardown() func() {
  fmt.Println("Run initialization")
  return func() {
    fmt.Println("Run cleanup")
  }
}

func main() {
  defer setupTeardown()() // <--------
  fmt.Println("Main function called")
}

Run initialization
Main function called
Run cleanup

Method Chaining¶

In [6]:
import "fmt"

type Person struct {
  Name string
  Age  int
}

func (p *Person) AddAge() *Person {
  p.Age++
  return p
}

func (p *Person) Rename(name string) *Person {
  p.Name = name
  return p
}

%%
p := Person{Name: "Aiden", Age: 30}
fmt.Println(p)
p = *p.AddAge().Rename("AIDEN")
fmt.Println(p)

{Aiden 30}
{AIDEN 31}

Error Handling¶

Try/catch implementation¶

  • Golang : Try, Catch, Finally by saldinata bobby
    • a new perspective on error handling in Go - an implementation which enables the use of try/catch blocks.
In [1]:
import (
  "fmt"
)

type Block struct {
  Try func()
  Catch func(Exception)
  Finally func()
}

type Exception interface{}

func Throw(up Exception) {
  panic(up)
}

func (tcf Block) Do() {
  if tcf.Finally != nil {
    defer tcf.Finally()
  }
  if tcf.Catch != nil {
    defer func() {
      if r:= recover(); r != nil {
        tcf.Catch(r)
      }
    }()
  }
  tcf.Try()
}

%%
fmt.Println("application started")
Block {
  Try: func() {
    fmt.Println("this is good")
    Throw("Oh, no...!!")
  },
  Catch: func(e Exception) {
    fmt.Printf("Caught %v\n", e)
  },
  Finally: func() {
    fmt.Println("finally keep runs")
  },
}.Do()
fmt.Println("application shutdown")

application started
this is good
Caught Oh, no...!!
finally keep runs
application shutdown

Understanding Panic¶

  • Understanding and Preventing Panics in Go by Martin Havelka
    • when a panic occurs, the current function stops executing, and the program starts unwinding the call stack, running deferred functions along the way.
    • to prevent panics, it's important to handle errors properly by checking return values and using defensive programming techniques.
In [4]:
func catchFn(e Exception) {
  fmt.Printf("Caught: %v\n", e)
}

func try(fn func()) {
  Block {
    fn, catchFn, nil,
  }.Do()
}

%%
Block {
  func() {
    fmt.Println("a sample block")
  }, catchFn, nil,
}.Do()

fmt.Println("\nNil Pointer Dereference")
Block {
  func() {
    var pointer *int
    fmt.Println(*pointer)
  }, catchFn, nil,
}.Do()

try(func() {
  var pointer *int
  if pointer != nil {
    fmt.Println(*pointer)
  } else {
    fmt.Println("Pointer is nil")
  }
})

fmt.Println("\nIndex Out of Range")
try(func() {
  arr := []int{1,2,3}
  fmt.Println(arr[4])
})
try(func() {
  arr := []int{1, 2, 3}
  index := 4
  if index >= 0 && index < len(arr) {
    fmt.Println(arr[index])
  } else {
    fmt.Println("Index out of range")
  }
})

fmt.Println("\nFailed Type Assertions")
try(func () {
  var i interface{} = "hello"
  num := i.(int)
  fmt.Println(num)
})
try(func () {
  var i interface{} = "hello"
  if num, ok := i.(int); ok {
    fmt.Println(num)
  } else {
    fmt.Println("Type assertion failed")
  }
})

fmt.Println("\nClosing Closed Channels")
try(func () {
  ch := make(chan int)
  close(ch)
  close(ch)
})

fmt.Println("\nDivision by Zero")
try(func () {
  x := 0
  y := 1 / x
  fmt.Println(y)
})

fmt.Println("\nInvalid Memory Address or Nil Pointer Dereference")
try(func () {
  var x *struct{Value int}
  fmt.Println(x.Value)
})

a sample block

Nil Pointer Dereference
Caught: runtime error: invalid memory address or nil pointer dereference
Pointer is nil

Index Out of Range
Caught: runtime error: index out of range [4] with length 3
Index out of range

Failed Type Assertions
Caught: interface conversion: interface {} is string, not int
Type assertion failed

Closing Closed Channels
Caught: close of closed channel

Division by Zero
Caught: runtime error: integer divide by zero

Invalid Memory Address or Nil Pointer Dereference
Caught: runtime error: invalid memory address or nil pointer dereference

JSON¶

mapstructure module¶

In [20]:
!go mod init ilima.xyz/20240511-2115
// https://github.com/mitchellh/mapstructure
!go get github.com/mitchellh/mapstructure@v1.5.0

go: /home/local/go.mod already exists
exit status 1

Conventional Usage¶

In [21]:
import "github.com/mitchellh/mapstructure"

func normalDecode() {
  type Person struct {
    Name   string
    Age    int
    Emails []string
    Extra  map[string]string
  }
  input := map[string]interface{}{
    "name":   "Foo",
    "age":    21,
    "emails": []string{"one@gmail.com", "two@gmail.com", "three@gmail.com"},
    "extra": map[string]string{
      "twitter": "Foo",
    },
  }
  var result Person
  err := mapstructure.Decode(input, &result)
  if err != nil {
    panic(err)
  }
  fmt.Printf("%#v\n", result)
}

func jsonDecode() {
  var jsonStr = `{
    "name": "Foo",
    "age": 21,
    "gender": "male"
  }`
  type Person struct {
    Name   string
    Age    int
    Gender string
  }
  m := make(map[string]interface{})
  err := json.Unmarshal([]byte(jsonStr), &m)
  if err != nil {
    panic(err)
  }
  var result Person
  err = mapstructure.Decode(m, &result)
  if err != nil {
    panic(err.Error())
  }
  fmt.Printf("%#v\n", result)
}

%%
normalDecode()
jsonDecode()

main.Person{Name:"Foo", Age:21, Emails:[]string{"one@gmail.com", "two@gmail.com", "three@gmail.com"}, Extra:map[string]string{"twitter":"Foo"}}
main.Person{Name:"Foo", Age:21, Gender:"male"}

Embedded Structures¶

In [24]:
type School struct {
  Name string
}
type Address struct {
  City string
}
type Person struct {
  School    `mapstructure:",squash"`
  Address  `mapstructure:",squash"`
  Email      string
}
func embeddedStructDecode() {
  input := map[string]interface{}{
    "Name": "A1",
    "City":  "B1",
    "Email": "C1",
  }
  var result Person
  err := mapstructure.Decode(input, &result)
  if err != nil {
    panic(err)
  }
  fmt.Printf("%s, %s, %s\n", result.Name, result.City, result.Email)
}

%%
embeddedStructDecode()

A1, B1, C1

Metadata¶

In [27]:
func metadataDecode() {
  type Person struct {
    Name   string
    Age    int
    Gender string
  }
  input := map[string]interface{}{
    "name":  "A1",
    "age":   1,
    "email": "B1",
  }
  var md mapstructure.Metadata
  var result Person
  config := &mapstructure.DecoderConfig{
    Metadata: &md,
    Result:   &result,
  }
  decoder, err := mapstructure.NewDecoder(config)
  if err != nil {
    panic(err)
  }
  if err = decoder.Decode(input); err != nil {
    panic(err)
  }
  fmt.Printf("value: %#v, keys: %#v, Unused keys: %#v, Unset keys: %#v\n", result, md.Keys, md.Unused, md.Unset)
}

%%
metadataDecode()

value: main.Person{Name:"A1", Age:1, Gender:""}, keys: []string{"Name", "Age"}, Unused keys: []string{"email"}, Unset keys: []string{"Gender"}

Avoiding Mapping of Null Values¶

In [29]:
func omitemptyDecode() {
  type School struct {
    Name string
  }
  type Address struct {
    City string
  }
  type Person struct {
    *School   `mapstructure:",omitempty"`
    *Address `mapstructure:",omitempty"`
    Age       int
    Email     string
  }
  result := &map[string]interface{}{}
  input := Person{Email: "C1"}
  err := mapstructure.Decode(input, &result)
  if err != nil {
    panic(err)
  }
  fmt.Printf("%+v\n", result)
}

func remainDataDecode() {
  type Person struct {
    Name  string
    Age   int
    Other map[string]interface{} `mapstructure:",remain"`
  }
  input := map[string]interface{}{
    "name":   "A1",
    "age":    1,
    "email":  "B1",
    "gender": "C1",
  }
  var result Person
  err := mapstructure.Decode(input, &result)
  if err != nil {
    panic(err)
  }
  fmt.Printf("%#v\n", result)
}

%%
omitemptyDecode()
remainDataDecode()

&map[Age:0 Email:C1]
main.Person{Name:"A1", Age:1, Other:map[string]interface {}{"email":"B1", "gender":"C1"}}

Custom Tags¶

In [30]:
func tagDecode() {
  type Person struct {
    Name string `mapstructure:"person_name"`
    Age  int    `mapstructure:"person_age"`
 }
  input := map[string]interface{}{
    "person_name": "A1",
    "person_age":  1,
  }
  var result Person
  err := mapstructure.Decode(input, &result)
  if err != nil {
    panic(err)
  }
  fmt.Printf("%#v\n", result)
}

%%
tagDecode()

main.Person{Name:"A1", Age:1}

Weakly Typed Parsing.¶

In [32]:
func weaklyTypedInputDecode() {
  type Person struct {
    Name   string
    Age    int
    Emails []string
  }
  input := map[string]interface{}{
    "name":   123,  // number => string
    "age":    "11", // string => number
    "emails": map[string]interface{}{}, // empty map => empty array
  }
  var result Person
  config := &mapstructure.DecoderConfig{
    WeaklyTypedInput: true,
    Result:           &result,
  }
  decoder, err := mapstructure.NewDecoder(config)
  if err != nil {
    panic(err)
  }
  err = decoder.Decode(input)
  if err != nil {
    panic(err)
  }
  fmt.Printf("%#v\n", result)
}

%%
weaklyTypedInputDecode()
//   - bools to string (true = "1", false = "0")
//   - numbers to string (base 10)
//   - bools to int/uint (true = 1, false = 0)
//   - strings to int/uint (base implied by prefix)
//   - int to bool (true if value != 0)
//   - string to bool (accepts: 1, t, T, TRUE, true, True, 0, f, F,
//     FALSE, false, False. Anything else is an error)
//   - empty array = empty map and vice versa
//   - negative numbers to overflowed uint values (base 10)
//   - slice of maps to a merged map
//   - single values are converted to slices if required. Each
//     element is weakly decoded. For example: "4" can become []int{4}
//     if the target type is an int slice.

main.Person{Name:"123", Age:11, Emails:[]string{}}

Error Handling¶

In [33]:
func decodeErrorHandle() {
  type Person struct {
    Name   string
    Age    int
    Emails []string
    Extra  map[string]string
  }
  input := map[string]interface{}{
    "name":   123,
    "age":    "bad value",
    "emails": []int{1, 2, 3},
  }
  var result Person
  err := mapstructure.Decode(input, &result)
  if err != nil {
    fmt.Println(err.Error())
  }
}

%%
decodeErrorHandle()

5 error(s) decoding:

* 'Age' expected type 'int', got unconvertible type 'string', value: 'bad value'
* 'Emails[0]' expected type 'string', got unconvertible type 'int', value: '1'
* 'Emails[1]' expected type 'string', got unconvertible type 'int', value: '2'
* 'Emails[2]' expected type 'string', got unconvertible type 'int', value: '3'
* 'Name' expected type 'string', got unconvertible type 'int', value: '123'

encoding/json¶

dynamic JSON parsing¶

In [35]:
import (
  "encoding/json"
  "fmt"
)

%%
// JSON data as a byte slice
jsonData := []byte(`{
  "name": "John Doe",
  "age": 30,
  "city": "New York",
  "hasCar": true,
  "languages": ["Go", "JavaScript"]
}`)

// Parse JSON into an empty interface
var result interface{}
err := json.Unmarshal(jsonData, &result)
if err != nil {
  fmt.Println("Error:", err)
  return
}

// Accessing dynamic JSON fields
dataMap, ok := result.(map[string]interface{})
if !ok {
  fmt.Println("Invalid JSON structure")
  return
}

// Accessing specific fields
name, nameExists := dataMap["name"].(string)
age, ageExists := dataMap["age"].(float64)
city, cityExists := dataMap["city"].(string)
hasCar, hasCarExists := dataMap["hasCar"].(bool)
languages, languagesExists := dataMap["languages"].([]interface{})

// Displaying parsed data
if nameExists {
  fmt.Println("Name:", name)
}

if ageExists {
  fmt.Println("Age:", int(age))
}

if cityExists {
  fmt.Println("City:", city)
}

if hasCarExists {
  fmt.Println("Has Car:", hasCar)
}

if languagesExists {
  fmt.Println("Languages:")
  for _, lang := range languages {
    fmt.Println(" -", lang)
  }
}

Name: John Doe
Age: 30
City: New York
Has Car: true
Languages:
 - Go
 - JavaScript

Test¶

In [9]:
func Add(a ...int) int {
  total := 0
  for i := range a {
    total += a[i]
  }
  return total
}

%test
func TestAdd(t *testing.T) {
  // case 1
  if res := Add(1, 2, 3); res != 6 {
    t.Errorf("Expected %d instead of %d", 6, res)
  }

  // case 2
  if res := Add(-1, -2); res != -3 {
    t.Errorf("Expected %d instead of %d", -3, res)
  }
}

=== RUN   TestAdd
--- PASS: TestAdd (0.00s)
PASS
In [ ]: