recentpopularlog in

dlkinney : go   53

afex/hystrix-go: Netflix's Hystrix latency and fault tolerance library, for Go
Hystrix is a great project from Netflix. "Hystrix is a latency and fault tolerance library designed to isolate points of access to remote systems, services and 3rd party libraries, stop cascading failure and enable resilience in complex distributed systems where failure is inevitable." I think the Hystrix patterns of programmer-defined fallbacks and adaptive health monitoring are good for any distributed system. Go routines and channels are great concurrency primitives, but don't directly help our application stay available during failures. hystrix-go aims to allow Go programmers to easily build applications with similar execution semantics of the Java-based Hystrix library.
distributed  go  golang  circuitbreaker 
2 days ago by dlkinney
gojektech/heimdall: An enhanced HTTP client for Go
Heimdall is an HTTP client that helps your application make a large number of requests, at scale. With Heimdall, you can: (1) Use a hystrix-like circuit breaker to control failing requests; (2) Add synchronous in-memory retries to each request, with the option of setting your own retrier strategy; (3) Create clients with different timeouts for every request; (4) All HTTP methods are exposed as a fluent interface.
http  go  golang  circuitbreaker 
2 days ago by dlkinney
pquerna/ffjson: faster JSON serialization for Go
ffjson generates static MarshalJSON and UnmarshalJSON functions for structures in Go. The generated functions reduce the reliance upon runtime reflection to do serialization and are generally 2 to 3 times faster. In cases where ffjson doesn't understand a Type involved, it falls back to encoding/json, meaning it is a safe drop in replacement. By using ffjson your JSON serialization just gets faster with no additional code changes.
github  golang  go  json  library  performance 
6 days ago by dlkinney
rs/zerolog: Zero Allocation JSON Logger
The zerolog package provides a fast and simple logger dedicated to JSON output. Zerolog's API is designed to provide both a great developer experience and stunning performance. Its unique chaining API allows zerolog to write JSON (or CBOR) log events by avoiding allocations and reflection. Uber's zap library pioneered this approach. Zerolog is taking this concept to the next level with a simpler to use API and even better performance. To keep the code base and the API simple, zerolog focuses on efficient structured logging only. Pretty logging on the console is made possible using the provided (but inefficient) zerolog.ConsoleWriter.
go  golang  json  logging  cbor 
7 days ago by dlkinney
Build your own OAuth2 Server in Go – Hacker Noon
Hello, in today’s article, I will show you how you can build your own OAuth2 server just like google, facebook, github etc. This will be very helpful if you want to build a production ready public or private API. So let’s get started.
golang  go  oauth2  tutorials  howto  authentication  article 
7 days ago by dlkinney
Watermill is a Golang library for working efficiently with message streams. It is intended for building event-driven applications, enabling event sourcing, RPC over messages, sagas and basically whatever else comes to your mind. You can use conventional pub/sub implementations like Kafka or RabbitMQ, but also HTTP or MySQL binlog if that fits your use case. It comes with a set of Pub/Sub implementations, which can be easily extended by your own implementations. Watermill is also shipped with a set of standard tools (middlewares) like instrumentation, poison queue, throttling, correlation and other tools used by every message-driven application. Our goal was to create a tool which is easy to understand, even by junior developers. It doesn't matter if you want to do Event-driven architecture, CQRS, Event Sourcing or just stream MySQL Binlog to Kafka. Watermill was designed to process hundreds of thousands of messages per second. Every component is built in a way that allows you to configure it for your needs. You can also implement your own middlewares for the router. Watermill is using proven technologies and has a strong unit and integration tests coverage for the critical areas.
golang  go  messaging  pubsub  agnostic 
7 days ago by dlkinney
Understanding Real-World Concurrency Bugs in Go (PDF)
In this paper, we conduct the first empirical study on Go concurrency bugs using six open-source, production- grade Go applications: Docker [13] and Kubernetes [36], two datacenter container systems, etcd [15], a distributed key-value store system, gRPC [19], an RPC library, and Cock- roachDB [10] and BoltDB [6], two database systems. In total, we have studied 171 concurrency bugs in these ap- plications. We analyzed the root causes of them, performed experiments to reproduce them, and examined their fixing patches. Finally, we tested them with two existing Go concurrency bug detectors (the only publicly available ones). To understand Go concurrency bugs and the comparison between message passing and shared memory, we propose to categorize concurrency bugs along two orthogonal dimen- sions: the cause of bugs and their behavior. Along the cause dimension, we categorize bugs into those that are caused by misuse of shared memory and those caused by misuse of message passing. Along the second dimension, we separate bugs into those that involve (any number of) goroutines that cannot proceed (we call them blocking bugs) and those that do not involve any blocking (non-blocking bugs). Surprisingly, our study shows that it is as easy to make con- currency bugs with message passing as with shared memory, sometimes even more. For example, around 58% of blocking bugs are caused by message passing. In addition to the viola- tion of Go’s channel usage rules (e.g., waiting on a channel that no one sends data to or close), many concurrency bugs are caused by the mixed usage of message passing and other new semantics and new libraries in Go, which can easily be overlooked but hard to detect.
golang  go  concurrency  bugs  research  paper 
7 days ago by dlkinney
Emitter: Scalable Real-Time Communication Across Devices
Emitter is a real-time communication service for connecting online devices. The Publish-Subscribe messaging API is built for speed and security.
communication  realtime  service  golang  go  messaging  mqtt  iot  internetofthings  mobile 
7 days ago by dlkinney
How to write a reverse-proxy with Go in 25 minutes - YouTube
In a world littered with containers, microservices, and virtual networks, the need for a reverse-proxy increases. I'll introduce you to the world of reverse-proxies by writing one -- live, and from scratch. Step by step, we'll discover the responsibilities of a reverse-proxy and how to fulfill them.
golang  go  http  reverseproxy  videos  presentations 
7 days ago by dlkinney
Home :: TinyGo - Go on Microcontrollers and WASM
TinyGo is a project to bring the Go programming language to microcontrollers and modern web browsers by creating a new compiler based on LLVM. You can compile and run TinyGo programs on several different microcontroller boards such as the BBC:Microbit and the Arduino Uno.
golang  go  microcontrollers  maker  arduino  microbit 
16 days ago by dlkinney
bloom42/rz-go: RIPZap - The fastest structured, leveled JSON logger for Go 📖
The rz package provides a fast and simple logger dedicated to JSON output. Rz's API is designed to provide both a great developer experience and stunning performance. Its unique functional API allows rz to write JSON (or CBOR) log events by avoiding allocations and reflection. Uber's zap and rs's zerolog libraries pioneered this approach. Rz is taking this concept to the next level with a simpler to use and safer API and even better performance. To keep the code base and the API simple, rz focuses on efficient structured logging only. Pretty logging on the console is made possible using the provided (but inefficient) rz.ConsoleFormatter.
go  golang  json  logging 
16 days ago by dlkinney
dvyukov/go-fuzz: Randomized testing for Go
Go-fuzz is a coverage-guided fuzzing solution for testing of Go packages. Fuzzing is mainly applicable to packages that parse complex inputs (both text and binary), and is especially useful for hardening of systems that parse inputs from potentially malicious users (e.g. anything accepted over a network).
github  go  golang  testing  fuzzing 
16 days ago by dlkinney
Using Golang to Build Microservices at The Economist: A Retrospective
Services written in Go was a key component of the new system that would enable The Economist to deliver scalable, high performing services and quickly iterate new products. Go's baked in concurrency and API support along with its design as a static, compiled language would enable a distributed eventing systems that could perform at scale. Testing support is also excellentOverall, The Economist team's experience with Go has been positive experience, and this has been one of the critical elements that has allowed the Content Platform to scale.
golang  go  economist  infoq  articles  contentmanagementsystem  cms  casestudies 
20 days ago by dlkinney
Writing Apps in Go and Swift · Young Dynasty
To write a C library in Go, we need to use cgo, the bridge between C and Go. For now, it’s enough just to know that the C package can convert Go values to and from C types, and vice-versa. If you want to dive-in a little deeper, the Go authors have written an excellent post about cgo on the Go Blog. As mentioned previously, to build a C archive, we need to create a main package and mark each method we want to export with a preceding //export cgo comment.
c  go  golang  programming  swift  ios 
23 days ago by dlkinney
GoCity - Code City metaphor for visualizing Go source code in 3D
GoCity is an implementation of the Code City metaphor for visualizing Go source code.
golang  3d  programming  visualization  code  go  maintainability 
25 days ago by dlkinney
esimov/caire at v1.2.0
Caire is a content aware image resize library based on Seam Carving for Content-Aware Image Resizing paper.
go  golang  library  images  graphics 
5 weeks ago by dlkinney
micro/go-micro: A microservice framework
Go Micro is a pluggable framework for micro service development. Go Micro provides the core requirements for distributed systems development including RPC and Event driven communication. The micro philosophy is sane defaults with a pluggable architecture. We provide defaults to get you started quickly but everything can be easily swapped out.
distributed  framework  golang  microservices  rpc  go 
5 weeks ago by dlkinney
Practical Cryptography With Go | Leanpub
This is a book about cryptography: how to communicate securely. There are several objectives that cryptography aims to solve: confidentiality, integrity, and authenticity. It also helps solve some other problems that come up in secure communications, but it’s important to remember that it isn’t a complete solution to security problems. In this book, we’ll look at how to build secure systems; some of the problems that cryptography does not solve will also be pointed out. This book will attempt to guide you in your attempt to understand how to use cryptography to secure your services, and illustrate it using the Go programming language.
go  golang  cryptography  security  programming  books  ebooks 
5 weeks ago by dlkinney
recoilme/pudge: Fast and simple key/value store written using Go's standard library
Fast and simple key/value store written using Go's standard library. Package pudge is a fast and simple key/value store written using Go's standard library. It presents the following: Supporting very efficient lookup, insertions and deletions; Performance is comparable to hash tables; Ability to get the data in sorted order, which enables additional operations like range scan; Select with limit/offset/from key, with ordering or by prefix; Safe for use in goroutines; Space efficient; Very short and simple codebase; Well tested, used in production
go  golang  keyvaluestores 
5 weeks ago by dlkinney
twpayne/chezmoi: Manage your dotfiles securely across multiple machines.
Manage your dotfiles securely across multiple machines. Declarative: you declare the desired state of files, directories, and symbolic links in your home directory and chezmoi updates your home directory to match that state. Flexible: your dotfiles can be templates (using text/template syntax). Predefined variables allow you to change behaviour depending on operating system, architecture, and hostname. Secure: chezmoi can retreive secrets from Bitwarden, LastPass, Vault, your Keychain (on macOS), and GNOME Keyring (on Linux). Robust: chezmoi updates all files and symbolic links atomically (using google/renameio) so you are never left with incomplete files that could lock you out, even if the update process is interrupted. Portable: chezmoi's configuration uses only visible, regular files and directories and so is portable across version control systems and operating systems. Transparent: chezmoi includes verbose and dry run modes so you can review exactly what changes it will make to your home directory before making them. Fast, easy to use, and familiar: chezmoi runs in fractions of a second and includes commands to make most operations trivial. You can use the version control system of your choice to manage your configuration, and many different formats (e.g. JSON, YAML, TOML, etc.) are supported for the configuration file.
dotfiles  golang  go  tools  system  administration  configuration  linux  macos 
5 weeks ago by dlkinney
cloudevents/sdk-go: Go SDK for CloudEvents (
Go SDK for CloudEvents ( NOTE: This SDK is still considered work in progress, things might (and will) break with every update.
go  golang  sdk  library  cloudevents  events  cncf 
5 weeks ago by dlkinney
VIDEO Go Modules for Package Maintainers - YouTube
Wondering how to make sure your Go package is backwards compatible with Deps? How about Go 1.8? In this video, Mark Bates will walk through the most common mistakes that package maintainers make when implementing Go modules, as well as how to properly handle forward and backwards compatibility for Dep, Go Modules, older versions of Go, and more.
go  golang  library 
6 weeks ago by dlkinney
Stop writing broken Go libraries
Do not hardcode the HTTP client. Do not introduce global behaviour. Return structs not interfaces. Use configuration structs to avoid modifying your APIs.
antipatterns  go  golang  bestpractices 
6 weeks ago by dlkinney
Accept Interfaces Return Struct in Go - My Code Smells!
The rule is, that if it walks like a duck and quacks as a duck, it's a duck, which means that we judge the particular struct by the way it behaves, not by how it is defined. You must realize that when a caller wants to use some function that returns anything, it can be either interested in a concrete type (expects a specific struct), or interested in a behavior (expect an interface). In the first case, you just need to return a struct, end of story. But in second, you should go back and see that interface is implemented when its functions are a part of the struct, so why don't you just return a compatible struct? A good approach to that situation would be to define what is it exactly that we expect from our input parameters. To do that, we define interfaces where they are used and limit them to the bare minimum that is required for our code to work. Unlike Java, in Go it's the caller that defines the interfaces it uses.
go  golang  bestpractices 
6 weeks ago by dlkinney
GitHub - esimov/pigo: Golang implementation of Pico face detection library.
Pigo is a face detection library implemented in Go based on Pixel Intensity Comparison-based Object detection paper.
detection  go  golang  machinelearning  ml  images  library 
6 weeks ago by dlkinney
Modules · golang/go Wiki · GitHub
A brief tour of other common functionality you might use: (go list -m all) View final versions that will be used in a build for all direct and indirect dependencies. (go list -u -m all) View available minor and patch upgrades for all direct and indirect dependencies. (go get -u) or (go get -u=patch) Update all direct and indirect dependencies to latest minor or patch upgrades. (go build ./...) or (go test ./...) Build or test all packages in the module when run from the module root directory. (go mod tidy) Prune any no-longer-needed dependencies from go.mod and add any dependencies needed for other combinations of OS, architecture, and build tags. (replace directive or gohack) Use a fork, local copy or exact version of a dependency. (go mod vendor) Optional step to create a vendor directory.
golang  go  modules  dependencies 
6 weeks ago by dlkinney
GitHub - alecthomas/gometalinter: Concurrently run Go lint tools and normalise their output
Concurrently run Go lint tools and normalise their output. The number of tools for statically checking Go source for errors and warnings is impressive. This is a tool that concurrently runs a whole bunch of those linters and normalises their output to a standard format.
go  golang  lint  formatting  style  tools 
7 weeks ago by dlkinney
Go Language - Web Application Secure Coding Practices
Go Language - Web Application Secure Coding Practices is a guide written for anyone who is using the Go Programming Language and aims to use it for web development. This book is collaborative effort of Checkmarx Security Research Team and it follows the OWASP Secure Coding Practices - Quick Reference Guide v2 (stable) release. The main goal of this book is to help developers avoid common mistakes while at the same time, learning a new programming language through a "hands-on approach". This book provides a good level of detail on "how to do it securely" showing what kind of security problems could arise during development.
books  go  golang  programming  security  guides  bestpractices 
7 weeks ago by dlkinney
Fountain codes and animated QR · divan's blog
It turned out, LT codes (LT stands for Luby Transform) are just one of the implementations of the broader family of codes called fountain codes. It’s a class of erasure codes that can produce a potentially infinite amount of blocks from the source message blocks (K), and it’s enough to receive slightly more than K encoded blocks to decode the message successfully. In coding theory, an erasure code is a forward error correction (FEC) code under the assumption of bit erasures (rather than bit errors), which transforms a message of k symbols into a longer message (code word) with n symbols such that the original message can be recovered from a subset of the n symbols. The fraction r = k/n is called the code rate. The fraction k’/k, where k’ denotes the number of symbols required for recovery, is called reception efficiency. The receiver can start receiving blocks from any point, receive blocks in any order, with any erasure probability – fountain codes will work as soon as you received K+ different blocks. That’s where name “fountain” comes from, actually – fountain’s water drops represent encoded blocks, and you just have to fill the bucket with enough drops, without actually caring which drops exactly you get. So the idea behind LT codes is relatively simple – encoder splits message into source blocks , and continuously creates encoded blocks that consist of 1, 2 or more randomly selected source blocks XOR-ed with each other. Block IDs used to construct each new encoded block are stored in it in an arbitrary way. Decoder, in its turn, collects all the encoded blocks (like a drops from a fountain) – some consisting from just 1 source block , some from 2 or more – and tries to restore new blocks by XORing them back with already decoded blocks. So when decoder sees the encoded block with just 1 source block – it does nothing else than adding it to the list of decoded blocks. Once it catches the block with 2 XOR-ed source blocks , it checks the IDs, and if one of them is already in the list of decoded blocks – easily restore it due to the property of XOR. The same applies to decoding encoded blocks with more than 2 source blocks – once you have all but one blocks decoded – just XOR it.
golang  go  fountaincode  erasurecode  google 
7 weeks ago by dlkinney
SliceTricks · golang/go Wiki
Slice tips. Copy. Cut. Delete. Delete without preserving order. Insert. Push-Pop. Unshift-Shift. Filtering without allocating. Reversing. Shuffling. Batching with minimal allocation. In-place deduplicate (comparable).
golang  go  slices 
9 weeks ago by dlkinney
Introducing gqlgen: a GraphQL Server Generator for Go - 99designs
What we needed was a type-safe system for APIs. GraphQL looked promising. As we explored it, however, we realized that there wasn’t a server approach out there that met all of our needs. So we developed our own, which we call gqlgen.
graphql  golang  go 
november 2018 by dlkinney
port APIs written with Go frameworks such as Gin to AWS Lambda and Amazon API Gateway
aws-lambda-go-api-proxy makes it easy to run Golang APIs written with frameworks such as Gin with AWS Lambda and Amazon API Gateway.
aws  go  golang  lambda  web  apigateway  labs 
october 2018 by dlkinney
GitHub - rcrowley/go-metrics: Go port of Coda Hale's Metrics library
Go port of Coda Hale's Metrics library: Create and update metrics. Periodically log every metric. Maintain all metrics along with expvars at /debug/metrics.
data  go  golang  metrics  operations  devops  observability 
october 2018 by dlkinney
Fastest JSON parser ever
json iterator. Fastest JSON parser ever. jsoniter (json-iterator) is fast and flexible JSON parser available in Java and Go. Good deal of the code is ported from dsljson, and jsonparser. Add import "" and replace json.Marshal with jsoniter.Marshal. Then the code should behave exactly the same, just much faster. Unlike easyjson or other json libaries, jsoniter does not rely on static code generation. Unmarshal, NewEncoder, NewDecoder they all works. Existing types implemented Marshaler or Unmarshaler interface will also work. Map with non-string key also work. Yes, everything just works.
go  golang  java  json  parsing  parser 
october 2018 by dlkinney
Casbin · An authorization library that supports access control models like ACL, RBAC, ABAC for Golang, Java, PHP and Node.js
An authorization library that supports access control models like ACL, RBAC, ABAC for Golang, Java, PHP and Node.js. Hybrid access control models. In Casbin, an access control model is abstracted into a CONF file based on the PERM metamodel (Policy, Effect, Request, Matchers). So switching or upgrading the authorization mechanism for a project is just as simple as modifying a configuration. Flexible policy storage. Besides memory and file, Casbin policy can be stored into lots of places. Currently, dozens of databases are supported, from MySQL, Postgres, Oracle to MongoDB, Redis, Cassandra, AWS S3. Cross-languages & cross-platforms. Casbin is implemented in Golang, Java, PHP and Node.js. All implementations share the same API and behaviors. You can learn Casbin once and use it everywhere.
authorization  java  javascript  nodejs  node  php  golang  go  accesscontrol  acl  rbac  abac 
september 2018 by dlkinney
GitHub - google/tink: Tink is a multi-language, cross-platform library that provides cryptographic APIs that are secure, easy to use correctly, and hard(er) to misuse.
Tink is a multi-language, cross-platform library that provides cryptographic APIs that are secure, easy to use correctly, and hard(er) to misuse. Java, C++, Go, Objective-C.
java  cpp  go  golang  objectivec  objc  swift  google  cryptography 
september 2018 by dlkinney
Bleve -- Bleve
full-text search and indexing for Go. Simple top-level API. Index any object in your data model. Override default mapping to customize behavior. Rich set of interfaces for extending the capabilities.
golang  text  search  indexing  go 
september 2018 by dlkinney
cheekybits/genny: Elegant generics for Go
Elegant generics for Go. Until the Go core team include support for generics in Go, genny is a code-generation generics solution. It allows you write normal buildable and testable Go code which, when processed by the genny gen tool, will replace the generics with specific types.
go  golang  generics  generator 
september 2018 by dlkinney
lestrrat/go-jsval: Validator toolset, aimed to be used with JSON Schema
The go-jsval package is a data validation toolset, with a tool to generate validators in Go from JSON schemas.
golang  go  json  jsonschema  schema 
september 2018 by dlkinney
Getting started with Onion Omega2 and Golang – Alvaro Viebrantz – Medium
The Omega2 is a powerful device made by the people of It's basically a computer that comes packed with built-in Wi-Fi, running a Linux Operating System based on OpenWRT, has many general purpose inputs and outputs, and comes with a powerful processor, all of this in a really small package, hoping that you build awesome IoT projects with it. The board is based on the MediaTek MT7688 processor (with a MIPS architecture) running at 580Mhz, comes with 64MB of RAM and 16MB of flash storage. It also has a version called Omega2 Plus that comes with the double of ram and storage. You can see more details at the Onion docs. I will show you here how to use Go programming language (or Golang) to program the Omega2. It's a simple, high-level language, with plenty of packages that can be used in your projects, awesome support for cross compiling and a small footprint. Our demo project, when compiled, generates a binary file just for the Omega architecture with a size of just 1MB, using external packages and more. So let's get started.
golang  go  onion  iot  internetofthings  onionio  walkthroughs  guides  tutorials 
august 2018 by dlkinney
Introducing Badger: A fast key-value store written purely in Go - Dgraph Blog
We have built an efficient and persistent log structured merge (LSM) tree based key-value store, purely in Go language. It is based upon WiscKey paper included in USENIX FAST 2016. This design is highly SSD-optimized and separates keys from values to minimize I/O amplification; leveraging both the sequential and the random performance of SSDs. We call it Badger. Based on benchmarks, Badger is at least 3.5x faster than RocksDB when doing random reads. For value sizes between 128B to 16KB, data loading is 0.86x - 14x faster compared to RocksDB, with Badger gaining significant ground as value size increases. On the flip side, Badger is currently slower for range key-value iteration, but that has a lot of room for optimization.
golang  go  keyvaluestores  embedded 
august 2018 by dlkinney
Dgraph — A Distributed, Fast Graph Database
Dgraph is an open source, scalable, distributed, highly available and fast graph database, designed from ground up to be run in production.
databases  graphdb  graphql  go  golang 
july 2018 by dlkinney
coreos/bbolt: An embedded key/value database for Go.
Bolt is a pure Go key/value store inspired by Howard Chu's LMDB project. The goal of the project is to provide a simple, fast, and reliable database for projects that don't require a full database server such as Postgres or MySQL. Since Bolt is meant to be used as such a low-level piece of functionality, simplicity is key. The API will be small and only focus on getting values and setting values. That's it.
github  golang  keyvaluestores  embedded  go 
july 2018 by dlkinney
Mobile · golang/go Wiki
The Go mobile subrepository adds support for mobile platforms (Android and iOS) and provides tools to build mobile applications. There are two strategies you can follow to include Go into your mobile stack: (1) Writing all-Go native mobile applications. (2) Writing SDK applications by generating bindings from a Go package and invoke them from Java (on Android) and Objective-C (on iOS). This article will contain step-by-step guides to explain how to achieve these strategies.
mobile  golang  android  ios  go 
july 2018 by dlkinney
How to Write Go Code - The Go Programming Language
This document demonstrates the development of a simple Go package and introduces the go tool, the standard way to fetch, build, and install Go packages and commands. The go tool requires you to organize your code in a specific way. Please read this document carefully. It explains the simplest way to get up and running with your Go installation.
programming  code  golang  conventions  idioms  go  tutorials 
february 2015 by dlkinney
Effective Go - The Go Programming Language
To write Go well, it's important to understand its properties and idioms. It's also important to know the established conventions for programming in Go, such as naming, formatting, program construction, and so on, so that programs you write will be easy for other Go programmers to understand. This document gives tips for writing clear, idiomatic Go code. It augments the language specification, the Tour of Go, and How to Write Go Code, all of which you should read first.
golang  idioms  go  programming  conventions 
february 2015 by dlkinney

Copy this bookmark:

to read