name: Toolbox Module

sort: 6

Core Toolbox Module

This module is based on the Dropwizard framework for Java. One user told me about this framework and it’s cool stuff. Issue 128 talked about this feature. I added more cool features. Here are the features: healthcheck, profiling, statistics and tasks.

Installation

  1. go get github.com/astaxie/beego/toolbox

Healthcheck

It can check the health status of your application. E.g.: To check if database is available:

  1. type DatabaseCheck struct {
  2. }
  4. func (dc *DatabaseCheck) Check() error {
  5.     if dc.isConnected() {
  6.         return nil
  7.     } else {
  8.         return errors.New("can't connect database")
  9.     }
  10. }

Then you can add it as a check item:

  1. toolbox.AddHealthCheck("database",&DatabaseCheck{})

After this you can send get request to /healthcheck:

  1. $ curl http://beego.vip:8088/healthcheck
  2. * deadlocks: OK
  3. * database: OK

It will return the database status accordingly.

Profiling

Monitoring the performance of running processes is a very good way to optimize performance and to look for issues in our application. E.g.: information of GC and goroutine.

Profile provides a easy entry point for you to debug the application. It uses the ProcessInput entry function to process the requests. Here are some debugging types:

  • lookup goroutine

    Print out the tasks of all goroutines which are currently running. You can easily see what all goroutines are doing.

    1.   goroutine 3 [running]:
    2.   runtime/pprof.writeGoroutineStacks(0x634238, 0xc210000008, 0x62b000, 0xd200000000000000)
    3.       /Users/astaxie/go/src/pkg/runtime/pprof/pprof.go:511 +0x7c
    4.   runtime/pprof.writeGoroutine(0x634238, 0xc210000008, 0x2, 0xd2676410957b30fd, 0xae98)
    5.       /Users/astaxie/go/src/pkg/runtime/pprof/pprof.go:500 +0x3c
    6.   runtime/pprof.(*Profile).WriteTo(0x52ebe0, 0x634238, 0xc210000008, 0x2, 0x1, ...)
    7.       /Users/astaxie/go/src/pkg/runtime/pprof/pprof.go:229 +0xb4
    8.   _/Users/astaxie/github/beego/toolbox.ProcessInput(0x2c89f0, 0x10, 0x634238, 0xc210000008)
    9.       /Users/astaxie/github/beego/toolbox/profile.go:26 +0x256
    10.   _/Users/astaxie/github/beego/toolbox.TestProcessInput(0xc21004e090)
    11.       /Users/astaxie/github/beego/toolbox/profile_test.go:9 +0x5a
    12.   testing.tRunner(0xc21004e090, 0x532320)
    13.       /Users/astaxie/go/src/pkg/testing/testing.go:391 +0x8b
    14.   created by testing.RunTests
    15.       /Users/astaxie/go/src/pkg/testing/testing.go:471 +0x8b2
    17.   goroutine 1 [chan receive]:
    18.   testing.RunTests(0x315668, 0x532320, 0x4, 0x4, 0x1)
    19.       /Users/astaxie/go/src/pkg/testing/testing.go:472 +0x8d5
    20.   testing.Main(0x315668, 0x532320, 0x4, 0x4, 0x537700, ...)
    21.       /Users/astaxie/go/src/pkg/testing/testing.go:403 +0x84
    22.   main.main()
    23.       _/Users/astaxie/github/beego/toolbox/_test/_testmain.go:53 +0x9c

  • lookup heap

    Print out information of current heap:

    1.   heap profile: 1: 288 [2: 296] @ heap/1048576
    2.   1: 288 [2: 296] @
  1.     # runtime.MemStats
  2.     # Alloc = 275504
  3.     # TotalAlloc = 275512
  4.     # Sys = 4069608
  5.     # Lookups = 5
  6.     # Mallocs = 469
  7.     # Frees = 1
  8.     # HeapAlloc = 275504
  9.     # HeapSys = 1048576
  10.     # HeapIdle = 647168
  11.     # HeapInuse = 401408
  12.     # HeapReleased = 0
  13.     # HeapObjects = 468
  14.     # Stack = 24576 / 131072
  15.     # MSpan = 4472 / 16384
  16.     # MCache = 1504 / 16384
  17.     # BuckHashSys = 1476472
  18.     # NextGC = 342976
  19.     # PauseNs = [370712 77378 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
  20.     # NumGC = 2
  21.     # EnableGC = true
  22.     # DebugGC = false

  • lookup threadcreate

    Print out information of threads:

    1.   threadcreate profile: total 4
    2.   1 @ 0x17f68 0x183c7 0x186a8 0x188cc 0x19ca9 0xcf41 0x139a3 0x196c0
    3.   #    0x183c7    newm+0x27            /Users/astaxie/go/src/pkg/runtime/proc.c:896
    4.   #    0x186a8    startm+0xb8            /Users/astaxie/go/src/pkg/runtime/proc.c:974
    5.   #    0x188cc    handoffp+0x1ac            /Users/astaxie/go/src/pkg/runtime/proc.c:992
    6.   #    0x19ca9    runtime.entersyscallblock+0x129    /Users/astaxie/go/src/pkg/runtime/proc.c:1514
    7.   #    0xcf41    runtime.notetsleepg+0x71    /Users/astaxie/go/src/pkg/runtime/lock_sema.c:253
    8.   #    0x139a3    runtime.MHeap_Scavenger+0xa3    /Users/astaxie/go/src/pkg/runtime/mheap.c:463
    10.   1 @ 0x17f68 0x183c7 0x186a8 0x188cc 0x189c3 0x1969b 0x2618b
    11.   #    0x183c7    newm+0x27        /Users/astaxie/go/src/pkg/runtime/proc.c:896
    12.   #    0x186a8    startm+0xb8        /Users/astaxie/go/src/pkg/runtime/proc.c:974
    13.   #    0x188cc    handoffp+0x1ac        /Users/astaxie/go/src/pkg/runtime/proc.c:992
    14.   #    0x189c3    stoplockedm+0x83    /Users/astaxie/go/src/pkg/runtime/proc.c:1049
    15.   #    0x1969b    runtime.gosched0+0x8b    /Users/astaxie/go/src/pkg/runtime/proc.c:1382
    16.   #    0x2618b    runtime.mcall+0x4b    /Users/astaxie/go/src/pkg/runtime/asm_amd64.s:178
    18.   1 @ 0x17f68 0x183c7 0x170bc 0x196c0
    19.   #    0x183c7    newm+0x27        /Users/astaxie/go/src/pkg/runtime/proc.c:896
    20.   #    0x170bc    runtime.main+0x3c    /Users/astaxie/go/src/pkg/runtime/proc.c:191
    22.   1 @

  • lookup block

    Print out information of block:

    1.   --- contention:
    2.   cycles/second=2294781025

  • start cpuprof

    Start recording cpuprof info into created file cpu-pid.pprof.

  • stop cpuprof

    Stop recording.

  • get memprof

    Start recording memprof into created file mem-pid.memprof

  • gc summary

    Check GC status:

    1.   NumGC:2 Pause:54.54us Pause(Avg):170.82us Overhead:177.49% Alloc:248.97K Sys:3.88M Alloc(Rate):1.23G/s Histogram:287.09us 287.09us 287.09us

Statistics

Look at this picture, what do you think? It’s cool, right? Toolbox module supports it.

Core Toolbox Module - 图1

How can I use the statistics? Add statistics like this:

  1. toolbox.StatisticsMap.AddStatistics("POST", "/api/user", "&admin.user", time.Duration(2000))
  2. toolbox.StatisticsMap.AddStatistics("POST", "/api/user", "&admin.user", time.Duration(120000))
  3. toolbox.StatisticsMap.AddStatistics("GET", "/api/user", "&admin.user", time.Duration(13000))
  4. toolbox.StatisticsMap.AddStatistics("POST", "/api/admin", "&admin.user", time.Duration(14000))
  5. toolbox.StatisticsMap.AddStatistics("POST", "/api/user/astaxie", "&admin.user", time.Duration(12000))
  6. toolbox.StatisticsMap.AddStatistics("POST", "/api/user/xiemengjun", "&admin.user", time.Duration(13000))
  7. toolbox.StatisticsMap.AddStatistics("DELETE", "/api/user", "&admin.user", time.Duration(1400))

Get statistics information:

  1. toolbox.StatisticsMap.GetMap(os.Stdout)

Here is the output:

  1. | requestUrl                                        | method     | times            | used             | max used         | min used         | avg used         |
  2. | /api/user                                         | POST       |  2               | 122.00us         | 120.00us         | 2.00us           | 61.00us          |
  3. | /api/user                                         | GET        |  1               | 13.00us          | 13.00us          | 13.00us          | 13.00us          |
  4. | /api/user                                         | DELETE     |  1               | 1.40us           | 1.40us           | 1.40us           | 1.40us           |
  5. | /api/admin                                        | POST       |  1               | 14.00us          | 14.00us          | 14.00us          | 14.00us          |
  6. | /api/user/astaxie                                 | POST       |  1               | 12.00us          | 12.00us          | 12.00us          | 12.00us          |
  7. | /api/user/xiemengjun                              | POST       |  1               | 13.00us          | 13.00us          | 13.00us          | 13.00us          |

Tasks

Tasks work very similarly to cron jobs. Tasks are used to run a job outside the normal request/response cycle. These can be adhoc or scheduled to run regularly. Examples include: Reporting memory and goroutine status, periodically triggering GC or cleaning up log files at fixed intervals.

Creating a new Task

To initialize a task implement https://godoc.org/github.com/astaxie/beego/toolbox#NewTask:

  1. tk1 := toolbox.NewTask("tk1", "0 12 * * * *", func() error {
  2.     fmt.Println("tk1")
  3.     return nil
  4. })

The NewTask signature:

  1. NewTask(tname string, spec string, f TaskFunc) *Task
  • tname: Task name
  • spec: Task format. See below for details.
  • f: The function which will be run as the task.

To implement this task, add it to the global task list and start it.

  1. toolbox.AddTask("tk1", tk1)
  2. toolbox.StartTask()
  3. defer toolbox.StopTask()

Testing the TaskFunc

Use the code below to test if the TaskFunc is working correctly.

  1. err := tk.Run()
  2. if err != nil {
  3.     t.Fatal(err)
  4. }

spec in detail

spec specifies when the new Task will be run. Its format is the same as that of traditional crontab:

  1. // The first 6 parts are:
  2. //       second: 0-59
  3. //       minute: 0-59
  4. //       hour: 1-23
  5. //       day: 1-31
  6. //       month: 1-12
  7. //       weekdays: 0-6（0 is Sunday）
  9. // Some special sign:
  10. //       *: any time
  11. //       ,: separator. E.g.: 2,4 in the third part means run at 2 and 4 o'clock
  12. //　　    －: range. E.g.: 1-5 in the third part means run between 1 and 5 o'clock
  13. //       /n : run once every n time. E.g.: */1 in the third part means run once every an hour. Same as 1-23/1
  14. /////////////////////////////////////////////////////////
  15. //    0/30 * * * * *                        run every 30 seconds
  16. //    0 43 21 * * *                         run at 21:43
  17. //    0 15 05 * * *                         run at 05:15
  18. //    0 0 17 * * *                          run at 17:00
  19. //    0 0 17 * * 1                          run at 17:00 of every Monday
  20. //    0 0,10 17 * * 0,2,3                   run at 17:00 and 17:10 of every Sunday, Tuesday and Wednesday
  21. //    0 0-10 17 1 * *                       run once every minute from 17:00 to 7:10 on 1st day of every month
  22. //    0 0 0 1,15 * 1                        run at 0:00 on 1st and 15th of each month and every Monday
  23. //    0 42 4 1 * *                          run at 4:42 on 1st of every month
  24. //    0 0 21 * * 1-6                        run at 21:00 from Monday to Saturday
  25. //    0 0,10,20,30,40,50 * * * *            run every 10 minutes
  26. //    0 */10 * * * *                        run every 10 minutes
  27. //    0 * 1 * * *                           run every one minute from 1:00 to 1:59
  28. //    0 0 1 * * *                           run at 1:00
  29. //    0 0 */1 * * *                         run at :00 of every hour
  30. //    0 0 * * * *                           run at :00 of every hour
  31. //    0 2 8-20/3 * * *                      run at 8:02, 11:02, 14:02, 17:02 and 20:02
  32. //    0 30 5 1,15 * *                       run at 5:30 of 1st and 15th of every month

Debug module (Already moved to utils module)

We always use print for debugging. But the default output is not good enough for debugging. Beego provides this debug module

  • Display() print result to console
  • GetDisplayString() return the string

It print key/value pairs. The following code:

Display("v1", 1, "v2", 2, "v3", 3)

will output:

2013/12/16 23:48:41 [Debug] at TestPrint() [/Users/astaxie/github/beego/toolbox/debug_test.go:13]

[Variables]
v1 = 1
v2 = 2
v3 = 3

For pointer type:

type mytype struct {
    next *mytype
    prev *mytype
}    

var v1 = new(mytype)
var v2 = new(mytype)

v1.prev = nil
v1.next = v2

v2.prev = v1
v2.next = nil

Display("v1", v1, "v2", v2)

The output result

2013/12/16 23:48:41 [Debug] at TestPrintPoint() [/Users/astaxie/github/beego/toolbox/debug_test.go:26]

[Variables]
v1 = &toolbox.mytype{
    next: &toolbox.mytype{
        next: nil,
        prev: 0x210335420,
    },
    prev: nil,
}
v2 = &toolbox.mytype{
    next: nil,
    prev: &toolbox.mytype{
        next: 0x210335430,
        prev: nil,
    },
}