Giving `dep` a try: Using the new Go dependency tool with piladb
piladb is a lightweight, RESTful database engine based on Stack data structures, written in Go. Store or fetch JSON-compatible elements from in-memory stacks via HTTP, so you can implement caching systems, message processing mechanisms, or anything that requires native-cloud stacks. Support development of piladb by trying our latest release, starring the Github repo, or following our Twitter account.
Earlier this week, the Go Package Committee announced the roadmap for dep, a tool that aims to become the standard solution to vendor libraries and apps dependencies within the Go ecosystem. The goal of this roadmap is to introduce dep into the Go toolchain for the beginning of version 1.10 development phase. However, this goal is not guaranteed, and it needs to be finally approved and embraced by the Go community — including the Go team, of course.
In this text, dep authors also ask for help to the community in all kind of areas:
Right now, we NEED PEOPLE to help shoulder the workload of writing docs, writing code, clarifying some design issues, and kicking the tires.
One of the clearest ways to provide help is by simply testing the tool and giving some feedback. piladb is a perfect example to become an early adopter and introduce dep:
- External dependencies are minimal, difficult to mess up with.
- Releases are pre-built, it won’t break current functionality.
- It’s a young project, good timing for vendoring changes.
The idea is to replace the excellent and simple gvt, used by piladb until now, with dep, and in the meantime get to know better its early design, features and limitations.
For giving a bit of context, these are the only third party dependencies that piladb is relying on:
- github.com/gorilla/mux: implements HTTP router. It uses github.com/gorilla/context as unique external dependency.
- github.com/mitchellh/go-homedir: used to get home directory in cross-compilation environments in a test file.
That’s all, really.
So the first thing we’ll do is to remove the current vendor directory from piladb:
⇒ cd $GOPATH/src/github.com/fern4lvarez/piladb
⇒ tree vendor
vendor
├── github.com
│ ├── gorilla
│ │ ├── context
│ │ │ ├── context.go
│ │ │ ├── doc.go
│ │ │ └── LICENSE
│ │ └── mux
│ │ ├── context_gorilla.go
│ │ ├── context_native.go
│ │ ├── doc.go
│ │ ├── LICENSE
│ │ ├── mux.go
│ │ ├── regexp.go
│ │ └── route.go
│ └── mitchellh
│ └── go-homedir
│ ├── homedir.go
│ └── LICENSE
└── manifest
⇒ rm -rf vendor
⇒ git commit -am "Remove gvt's vendor directory"At this point, if we push these changes and someone go get's this repo, it would download the latest version of these dependencies and store them in their $GOPATH. Let’s then install dep and inspect it:
⇒ go get -u github.com/golang/dep/...
⇒ dep -h
Usage: dep <command>Commands: init Initialize a new project with manifest and lock files
status Report the status of the project's dependencies
ensure Ensure a dependency is safely vendored in the project
remove Remove a dependency from the project
Following short dep documentation, we want now to initialize a new project using dep init, and this is what we get:
⇒ dep init
Cached github.com/gorilla/mux
⇒ dep status
PROJECT CONSTRAINT VERSION REVISION LATEST PKGS USED
github.com/gorilla/context * v1.1 a85d2e5 v1.1 1
github.com/gorilla/mux * v1.3.0 392c28f v1.3.0 1
github.com/mitchellh/go-homedir * branch master b8bc1bf b8bc1bf 1
⇒ cat lock.json
{
"memo": "56e9e2eb8dff4ae328332bf64aba30efaed5e2351b1dd942d089b814a3eac77c",
"projects": [
{
"name": "github.com/gorilla/context",
"version": "v1.1",
"revision": "a85d2e53ba63bdea074dbbbb5983f0516974e87b",
"packages": [
"."
]
},
{
"name": "github.com/gorilla/mux",
"version": "v1.3.0",
"revision": "392c28fe23e1c45ddba891b0320b3b5df220beea",
"packages": [
"."
]
},
{
"name": "github.com/mitchellh/go-homedir",
"branch": "master",
"revision": "b8bc1bf767474819792c23f32d8286a45736f1c6",
"packages": [
"."
]
}
]
}
⇒ cat manifest.json
⇒ tree vendor
vendor
└── github.com
├── gorilla
│ ├── context
│ │ ├── context.go
│ │ ├── context_test.go
│ │ ├── doc.go
│ │ ├── LICENSE
│ │ └── README.md
│ └── mux
│ ├── bench_test.go
│ ├── context_gorilla.go
│ ├── context_gorilla_test.go
│ ├── context_native.go
│ ├── context_native_test.go
│ ├── doc.go
│ ├── LICENSE
│ ├── mux.go
│ ├── mux_test.go
│ ├── old_test.go
│ ├── README.md
│ ├── regexp.go
│ └── route.go
└── mitchellh
└── go-homedir
├── homedir.go
├── homedir_test.go
├── LICENSE
└── README.md6 directories, 22 files
So we’ve got few things going on here:
depcreated alock.jsonandmanifest.jsonfiles in the root directory. Former contains meta-information about the vendored dependencies, latter is an empty JSON file. Note thatgvtonly had onemanifestfile insidevendor/directory, not in root.- Unlike
gvt,depcopied the dependencies code invendor/including tests and markdown files. I’m not sure about the motivation for this decision, but I expect it might change duringdep's development process. - From these dependencies, github.com/gorilla/mux is the only package with proper labeled, versioned releases. Thus,
depinstalled by default the latest stable release, v1.3.0, instead of checking out latest master. In turn, v1.1 of github.com/gorilla/context is used too.
These are no big deal for piladb, but I’d like to keep using the very same version that I was using with gvt, so apparently I need to dig into dep ensure now:
⇒ dep ensure -h
Usage: dep ensure [spec...]Ensure is used to fetch project dependencies into the vendor folder, as well as
to set version constraints for specific dependencies. It takes user input,
solves the updated dependency graph of the project, writes any changes to the
manifest and lock file, and places dependencies in the vendor folder.Package spec:<path>[:alt location][@<version specifier>]Examples:dep ensure Populate vendor from existing manifest and lock
dep ensure github.com/pkg/foo@^1.0.1 Update a specific dependency to a specific versionFor more detailed usage examples, see dep ensure -examples.Flags: -examples print detailed usage examples (default: false)
-n dry run, don't actually ensure anything (default: false)
-override specify an override constraint spec (repeatable) (default: <none>)
-update ensure dependencies are at the latest version allowed by the manifest (default: false)
-v enable verbose logging (default: false)
I find dep ensure -examples very interesting, as it shows extended usage examples, making emphasis in SemVer notations and version constraints using tilde or caret, which are widely used in the npm ecosystem, but are not that common in Go package managers.
⇒ dep ensure github.com/gorilla/mux@master
⇒ cat lock.json
{
"memo": "f1b9d5d8b37b8fe02b788694611b16a4fd8e7097edf980fdb1af70d086efc022",
"projects": [
{
"name": "github.com/gorilla/context",
"version": "v1.1",
"revision": "a85d2e53ba63bdea074dbbbb5983f0516974e87b",
"packages": [
"."
]
},
{
"name": "github.com/gorilla/mux",
"branch": "master",
"revision": "599cba5e7b6137d46ddf58fb1765f5d928e69604",
"packages": [
"."
]
},
{
"name": "github.com/mitchellh/go-homedir",
"branch": "master",
"revision": "b8bc1bf767474819792c23f32d8286a45736f1c6",
"packages": [
"."
]
}
]
}
⇒ cat manifest.json
{
"dependencies": {
"github.com/gorilla/mux": {
"branch": "master"
}
}
}
⇒ git commit -am "dep: ensure master for github.com/gorilla/mux"We can see that in lock.json the metadata for github.com/gorilla/mux was updated, pointing to master branch, and creating an entry in manifest.json file, specifying the new dependency. So now I assume whenever master branch is updated on the upstream, a dep ensure -update github.com/gorilla/mux will handle that.
Since github.com/gorilla/mux and github.com/gorilla/context are not connected as explicit dependencies, the latter wasn’t updated in the previous action, so it’s something I also need to do manually:
⇒ dep ensure github.com/gorilla/context@master
⇒ cat manifest.json
{
"dependencies": {
"github.com/gorilla/context": {
"branch": "master"
},
"github.com/gorilla/mux": {
"branch": "master"
}
}
}⇒ dep status
PROJECT CONSTRAINT VERSION REVISION LATEST PKGS USED
github.com/gorilla/context branch master v1.1 a85d2e5 08b5f42 1
github.com/gorilla/mux branch master branch master 599cba5 599cba5 1
github.com/mitchellh/go-homedir * branch master b8bc1bf b8bc1bf 1
⇒ git commit -am "dep: ensure master for github.com/gorilla/context"
With this, I know that both dependencies are up-to-date with master, and I’m one command away from updating them in case of changes in upstream.
So far dep looks good, but it has a long way ahead and I’m sure things will change in its functionality, and design over this process. Probably you are a reader from the future and see how dep evolved over the time, but remember that as of this writing the status is as follows:
Alpha. Functionality is known to be broken, missing or incomplete. Command and file format changes are still planned. (…).
Regarding piladb, the dependencies management here is so trivial that playing around with dep is a no-brainer. I created a PR with these changes, where we’ll consider if adopting it now, or giving it a bit more time, until authors consider it is usable.
Happy Friday!
