recentpopularlog in
« earlier  
GitHub - qri-io/qri: you're invited to a data party!
qri is a global dataset version control system (GDVCS) built on the distributed web

Breaking that down:

global so that if anyone, anywhere has published work with the same or similar datasets, you can discover it.
Specific to datasets because data deserves purpose-built tools
version control to keep data in sync, attributing all changes to authors
On the distributed web to make all of the data published on qri simultaneously available, letting peers work on data together.
qri  data  decentralized  distributed  versioncontrol 
6 hours ago
HighwayHash: Fast hashing at over 10 GB/s per core in Golang
HighwayHash is an approximately 5x faster SIMD hash function as compared to SipHash which in itself is a fast and ‘cryptographically strong’ pseudo-random function designed by Aumasson and Bernstein.
A new way of mixing inputs with Intel’s AVX2 multiply and permute instructions is exploited by HighwayHash to achieve its performance. The multiplications are 32x32-bit giving 64-bit wide results and are therefore infeasible to reverse. Additionally permuting equalizes the distribution of the resulting bytes.
On a Skylake CPU (3.0 GHz Xeon Platinum 8124M) the table below shows how HighwayHash compares to other hashing techniques for 5 MB-messages (single core performance, all Golang implementations, see benchmark).
BenchmarkHighwayHash 11986.98 MB/s
BenchmarkSHA256_AVX512 3552.74 MB/s
BenchmarkBlake2b 972.38 MB/s
golang  hash  performance 
15 hours ago
Raspberry Pi 3 Raspbian Primer - DEV Community 👩‍💻👨‍💻
More goodies:

There’s no official VS Code releases, but there’s community provided binaries
PowerShell Core
Install Rust via rustup: curl -sSf | sh
.Net Core for ARM (2.2.2 and/or 3.0.0)
Raspi  sysadmin 
17 hours ago
Relentless Competition Drives Down Ethernet Switch Costs
Four years ago, when 100 Gb/sec started to trickle into datacenters, often for network backbones linking datacenters but sometimes for network fabrics linking machines, a port cost about $3,000 a pop. If you do the math on the market data from IDC, it has dropped by more than a factor of 10X, averaging $288 per port. If you use a quad cable splitter, as some datacenters do from their top of racks, then the cost of a 25 Gb/sec port is around $72 a pop. At these prices, it is no wonder that 100 Gb/sec port shipments were almost triple year-on-year in the fourth quarter, to 3.8 million units, and revenues were up 58.5 percent to $1.1 billion. Sales of switches with real 25 Gb/sec and 50 Gb/sec ports, which IDC lumps together into one category, were up by 2.75X to $342 million, pushing 3 million ports, up by the same amount. The cost per port on these switches, assuming an even split of 25 Gb/sec and 50 Gb/sec switches, is around $115.
trend  ethernet  bandwidth 
21 hours ago
USB 3.1 to SATA Adapter Cable for 2.5" Drives |
USB 3.1 (10Gbps) Adapter Cable for 2.5" SATA Drives
Connect a 2.5" SATA SSD/HDD to your computer using this USB 3.1 Gen 2 ultra-portable cable
SSD  USB3.1Gen2  local 
NILM 2018: On the Feasibility of Generic Deep Disaggregation for Single-Load Extraction - YouTube
NILM (or disaggregation) is a growing research field which began in 1985 with a report written by George W. Hart (MIT) for Electric Power Research Institute (EPRI). NILM is used to discern what electrical loads (e.g., appliances) are running within a home/building using only the aggregate power meter. Why? To help occupants understand how they and their appliance use energy so that they could conserve to either save money, the environment, or both.
NILM  video 
build/ at master · golang/build · GitHub
Introducing Stargz

Fortunately, we can fix the fact that tar.gz files are unindexed and unseekable, while still making the file a valid tar.gz file by taking advantage of the fact that two gzip streams can be concatenated and still be a valid gzip stream. So you can just make a tar file where each tar entry is its own gzip stream.

We introduce a format, Stargz, a Seekable tar.gz format that's still a valid tar.gz file for everything else that's unaware of these details.

In summary:

That traditional *.tar.gz format is: Gzip(TarF(file1) + TarF(file2) + TarF(file3) + TarFooter))
Stargz's format is: Gzip(TarF(file1)) + Gzip(TarF(file2)) + Gzip(TarF(file3_chunk1)) + Gzip(F(file3_chunk2)) + Gzip(F(index of earlier files in magic file), TarFooter), where the trailing ZIP-like index contains offsets for each file/chunk's GZIP header in the overall stargz file.
This makes images a few percent larger (due to more gzip headers and loss of compression context between files), but it's plenty acceptable.
FUSE  containers  filesystem 
nilmtk/ at master · nilmtk/nilmtk · GitHub
NILMTK: Non-Intrusive Load Monitoring Toolkit

Non-Intrusive Load Monitoring (NILM) is the process of estimating the energy consumed by individual appliances given just a whole-house power meter reading. In other words, it produces an (estimated) itemised energy bill from just a single, whole-house power meter.

NILMTK is a toolkit designed to help researchers evaluate the accuracy of NILM algorithms.

As of June 2018, NILMTK is being revived! Although no major changes are expected in the coming months, the codebase is slowly being updated to work properly with the current Python ecosystem, especially to modern versions of our major dependency, Pandas. It may take time for the NILMTK authors to get back to you regarding queries/issues. However, you are more than welcome to propose changes, support!
electronics  power  NILM 
Which Safari extensions do you use? – The Eclectic Light Company
I only use two Safari extensions now, both of which I strongly recommend. The first is Jeff Johnson’s unique StopTheMadness, which fixes all those blocked features, and Better, which protects my privacy from trackers. The trouble with them both is that they’re so good at their jobs that I no longer notice how much they do for me: like all the best extensions, they just get on with it. When I want to copy or use Safari’s contextual menu, StopTheMadness just works as you’d expect. And I haven’t seen any tracking ads for, well, since I installed Better.
safari  extensions  webbrowsing 
We Built a Collaborative Documentation Site. Deploy Your Own With the Push of a Button.
Library is a wiki powered by a Google Docs backend. When you connect Library to a shared folder or team drive, it will traverse the documents in the folders and create your site content. Documents in Library are searchable, taggable, and can be grouped by desks or categories.
nyt  opensource  documentation 
The HoTT Book | Homotopy Type Theory
Homotopy type theory is a new branch of mathematics that combines aspects of several different fields in a surprising way. It is based on a recently discovered connection between homotopy theory and type theory. It touches on topics as seemingly distant as the homotopy groups of spheres, the algorithms for type checking, and the definition of weak ∞-groupoids. Homotopy type theory offers a new “univalent” foundation of mathematics, in which a central role is played by Voevodsky’s univalence axiom and higher inductive types. The present book is intended as a first systematic exposition of the basics of univalent foundations, and a collection of examples of this new style of reasoning — but without requiring the reader to know or learn any formal logic, or to use any computer proof assistant. We believe that univalent foundations will eventually become a viable alternative to set theory as the “implicit foundation” for the unformalized mathematics done by most mathematicians.
book  ebook  math  Homotopy  Type  Theory 
Meet the Adobe I/O Team: Meryll Blanchet on Cloud Native Applications
What tools and technologies do you work with?
Our SDK comes with both back-end and front-end features. The back-end pieces are made of npm libraries that are used by I/O Runtime actions and sequences, so server-side JavaScript in both cases.
We have chosen to build our sample CNAs on top of React. But developers will be free to use their favorite front-end library for their own apps.
We have a few open source dependencies, such as Parcel to build the UI, and Express to facilitate local development.
Last but not least, our CI/CD is based on Jenkins, and we use Mocha for our unit tests. We also plan to integrate with the rest of the I/O ecosystem like CLI, Console, or Events.
architecture  cloud  CNA  Adobe 
2 days ago
All You Need to Know About UASP | StarTech.Blog
This is made possible due to the way UASP functions in comparison to USB BOT. UASP processes transfers in parallel, delivering data signals (Commands, Status, and Data In & Out) in dedicated “pipes”. BOT on the other hand, requires each signal to travel on the same “pipe”, taking turns, thus slowing the file transfer process.
usb  storage  UASP 
3 days ago
How to Organize Your Messy Albums on on macOS with Some SQL and Image Processing
While the SQL is cool enough, it does not allow you to have write access to iCloud. The current PhotoKit on MacOS only allows read-only access. I hope there was an API, so I could do some of the stuff programatically, especially removing the bad quality Photos automatically. My long-term plan is to learn some iOS programming and do these kind of analysis in the app by requesting metadata and thumbnails and, maybe even work with some MLKit magic to cluster my photos.
MacOS  sqlite  OpenCV  sql  sysadmin 
3 days ago
GitHub - trailofbits/algo: Set up a personal IPSEC VPN in the cloud
Apple Devices

WireGuard is used to provide VPN services on Apple devices. Algo generates a WireGuard configuration file, wireguard/<username>.conf, and a QR code, wireguard/<username>.png, for each user defined in config.cfg.

On iOS, install the WireGuard app from the iOS App Store. Then, use the WireGuard app to scan the QR code or AirDrop the configuration file to the device.

On macOS Mojave or later, install the WireGuard app from the Mac App Store. WireGuard will appear in the menu bar once you run the app. Click on the WireGuard icon, choose Import tunnel(s) from file..., then select the appropriate WireGuard configuration file. Enable "Connect on Demand" by editing the tunnel configuration in the WireGuard app.

Installing WireGuard is a little more complicated on older version of macOS. See Using macOS as a Client with WireGuard.

If you prefer to use the built-in IPSEC VPN on Apple devices, then see Using Apple Devices as a Client with IPSEC.
linux  vpn  apple  ios 
3 days ago
How to make your code fast and asynchronous with Python and Sanic
Sanic is a very flask-like open-source Python web server and web framework with more than 10K stars that’s written to go fast. It allows the usage of async/await syntax added in Python 3.5 (read more), which makes your code non-blocking and speedy.
Sanic has pretty good documentation and it’s maintained by the community, for the community.
The goal of the project is to provide a simple way to get a highly performant HTTP server up and running that is easy to build, to expand, and ultimately to scale.
python  sanic  webdev  httpd 
3 days ago
Visualizing Complex Functions | Kevin Van Kessel

Each pixel to be plotted represents a point on the complex plane (
). This complex number is fed through a function that transforms it (
). This output is represented in polar coordinates (
). The pixel’s hue is mapped to the new angle (
), and the pixel’s lightness is mapped to the new magnitude (
functions  graph  visualization 
3 days ago
Myspace, Once the King of Social Networks, Lost Years of Data From Its Heyday - The New York Times
He and others warn this will be a recurring story, too. “Someday, this will happen to Facebook, Instagram, Tumblr, etc.,” Cory Doctorow, the author and co-editor of the tech site Boing Boing, wrote in a post. “Don’t trust the platforms to archive your data.”
cloud  outage  myspace 
3 days ago
Gast, Beniget! Use-Def chains for Python Programs
In spite of its dynamic nature, the Python language has been granted a rich ecosystem of Abstract Syntax Tree(AST) analysis and transformations. Off the top of my head, I can already cite:

Pyflake, a Python linter;
Pylint, a Python linter and more;
TorchScript, a DSL embedded into Python for PyTorch
autograph another DSL embedded into Python, for Tensorflow;
Pythran (of course!);
PEP-511 even proposed to normalize an API for code transformers-it got rejected.
And there probably are plenty of other users of the ast module from the Python standard library.

If you're using the ast module, then you should consider using the great (and home-backed) gast package. It offers a common tree structure to deal with the different Python version and their changes. In other words, if you can transform or analyze gast AST, then you can handle most Python versions.

Not convinced? It's already used by Tensorflow and Pythran. And Beniget :-)
Python  AST 
3 days ago
HP LaserJet 4050 Printer
Increased printing and networking performance at no additional cost with new 133-MHz processor, twice the standard memory, 1200-dpi output at full engine speed, and optional nwtworking with 600N (EIO) HP JetDirect internal print servers for Ethernet 10BaseT, 10Base2, 100BaseTX, LocalTalk and Token Ring networks.
HP4050N  printer  local 
3 days ago
GitHub - jfcoz/postgresqltuner: Simple script to analyse your PostgreSQL database configuration, and give tuning advice is a simple script to analyse your PostgreSQL database. It is inspired by
postgres  performance 
3 days ago
Myspace loses all content uploaded before 2016 | Technology | The Guardian
More than 50m tracks from 14 million artists have been lost, including songs that led to the rise of the “Myspace Generation” cohort of artists, such as Lily Allen, Arctic Monkeys and Yeasayer. As well as music, the site has also accidentally deleted pictures and videos stored on its servers.


Even though many users had deserted Myspace by the end of the noughties for newer social networks such as Facebook, the site retained a significant user base well into this decade as musicians had cultivated their fan followings on it. A disastrous relaunch in 2013 led to most bands having to rebuild their communities from scratch.
cloud  outage  MySpace 
4 days ago
Text Data Visualization in Python – Kanoki
You have seen how you can make good insight of your data using Scattertext in an easy and flexible without much of efforts. There are numerous other examples which can be found on their github page here. it provides a wide range of other ways to visualize your text data like Visualizing term association, Visualizing Empath topics and categories, Ordering Terms by Corpus Characteristictness, Document based scatter plots and many other advance usage of Text Data Visualization
text  dataviz  python 
4 days ago
HEIC Yeah - The Hacker Factor Blog
Fortunately, I did eventually find another solution. There's a different library (not based on Nokia's code) called libheif, and it's only dependent on libde265 (by the same author). Both of these are licensed as GNU Lesser General Public License (LGPL). That means that I can use it for commercial purposes as long as I don't integrate it into my code. (My code loads these as shared libraries if they exist on the system. And if they don't exist, then my code happily ignores the file format. I also use dlsym to access all functions, so there is zero LGPL code in my code.)
4 days ago
GitHub - facebookincubator/magma: Platform for building access networks and modular network services
Magma is an open-source software platform that gives network operators an open, flexible and extendable mobile core network solution. Magma enables better connectivity by:

Allowing operators to offer cellular service without vendor lock-in with a modern, open source core network
Enabling operators to manage their networks more efficiently with more automation, less downtime, better predictability, and more agility to add new services and applications
Enabling federation between existing MNOs and new infrastructure providers for expanding rural infrastructure
Allowing operators who are constrained with licensed spectrum to add capacity and reach by using Wi-Fi and CBRS
Magma Architecture

The figure below shows the high-level Magma architecture. Magma is 3GPP generation (2G, 3G, 4G or upcoming 5G networks) and access network agnostic (cellular or WiFi). It can flexibly support a radio access network with minimal development and deployment effort.

Magma has three major components:
cellular  mobile  network  facebook 
4 days ago
What Working At Stripe Has Been Like | Kalzumeus Software
I joined Stripe two years ago to make starting an Internet business easier, mostly by work on Stripe Atlas. After founding four small software companies I wanted peers to deal with less nonsense, either by productizing a solution to it or by writing up the things I wish I had known back in 2006 when I started. In the same spirit, here’s what I didn’t know prior to joining Stripe.
stripe  culture  finance  work 
4 days ago
Editorial by Peter Biľak (Works That Work magazine)
When I told my friends that this was going to be the last issue of Works That Work, most of them had more or less the same reaction: ‘I am so sorry to hear that.’ People like stability, the security of being able to keep things as they are in defiance of the basic law of nature which says that everything that has a beginning also has an end. Once in a while, however, we have the luxury of choosing an end, planning a conclusion that is timely, satisfying, even joyful.The start-up culture that we live in celebrates new things like launches, releases and births, choosing to ignore the less glorious endings of projects, products and lives. I believe we should be intentional about designing ends as well as beginnings. The plan for WTW has always been to publish 10 issues and to wrap up the project on a high, while we are still enjoying a thriving community of readers. In these 10 issues we have been defining a vision of design that I believe in, design that doesn’t just make things pretty, but design that brings lasting positive changes.
design  lifecycle  ending  rank:1 
5 days ago
Gauthier Roussilhe | Digital guide to low tech
Tools and publications

To find your "green" hosting service:

To make your website static:

Compress your media:

reSmushit (image)
Handbrake (video)
To probe the power consumption of your site:

Web Energy Archive (repository)
To test the speed of your site:

Pingdom Tools
Webpage Test
PageSpeed ​​Insights
To generate a radically low-tech site:

To enter the scientific publications on the calculation of the electricity consumed by the digital industry:

Start slowly with James Christie "Sustainable Web Design."
Calculation Method Proposed by Cody Taylor and Jonathan Koomey "Estimating Energy Use and Greenhouse Gas Emissions from Internet Advertising" (2008).
Review of calculation methods with Vlad Coroama and Lorenz Hilty "Assessing Internet energy intensity: A review of methods and results" (2014 ).
Understand usage peaks and social practices of energy with Janine Molley et al. "Digitization, Energy and Data Demand: The Impact of Internet Traffic on Overall and Peak Electricity Consumption" (2018).
webdev  energy  minimalism  design  tools  reference 
5 days ago
Microjs: Fantastic Micro-Frameworks and Micro-Libraries for Fun and Profit!
Fantastic Micro-Frameworks and
Micro-Libraries for Fun and Profit!
How much library code do you really need — 50K? 100K? 150K? More? How much of that do you really use?
Sure, we all love our favorite monolithic frameworks, and sometimes we even use them fully. But how often do we reach for the ride-on John Deere tractor with air conditioning and six-speaker sound system, when a judiciously applied pocketknife would do the trick better, faster, slicker?
Micro-frameworks are definitely the pocketknives of the JavaScript library world: short, sweet, to the point. And at 5k and under, micro-frameworks are very very portable. A micro-framework does one thing and one thing only — and does it well. No cruft, no featuritis, no feature creep, no excess anywhere. helps you discover the most compact-but-powerful microframeworks, and makes it easy for you to pick one that’ll work for you.
Want to add your own? Fork this site on GitHub, add your framework to data.js and submit a pull request.
Javascript  reference  resources  frameworks  tools 
5 days ago
GitHub - JXA-Cookbook/JXA-Cookbook: Cookbook for JavaScript for Automation in Mac OS X Yosemite
JavaScript for Automation is a new thing in Mac OS X Yosemite. It allows you to control applications using JavaScript language. How awesome!

However, its documentation covers the basics of using JavaScript to interact with the Open Scripting Architecture. For many tasks in scripting, there's a lot to figure out, and many different documentations to refer to.

So I created this JavaScript for Automation cookbook that features a lot of examples, such as:

How to display alert, prompt and confirm dialogs.
How to run shell scripts.
How to create a command-line script (shebang script) and read argv arguments, how to create a Mac OS X service, or how to invoke the REPL.
How to use Browserify to be able to require npm modules in JXA.
Where it makes sense, I try to link to the official documentation as much as possible so you can dive deeper.

For ease of editing and contribution, the cookbook's contents are on the wiki.
MacOS  automation  javascript  cookbook  reference 
5 days ago
Bringing Byte-Level Binary Data Structures to JavaScript with Construct-JS | Hacker News
sheetjs 1 hour ago [-]

First off, great to see interest in binary data in JS :)
We looked into this problem years ago as part of our spreadsheet parser/writer library (open source version and ultimately opted for lower-level functions that act on Buffers/ArrayBuffers/Arrays. Performance-wise, creating a new ArrayBuffer / Buffer for each field (which you are currently doing in is incredibly expensive. When we did performance tests years ago, allocating in 2KB blocks and manually orchestrating writes was nearly 10x faster than individual field-level allocations and concatenations both in the browser and in NodeJS.
Ironically, the ZIP file example referenced in the README alludes to another pitfall in the approach: the actual DEFLATE algorithm used in compression actually requires unaligned bit writes. See section 5.5 of the current APPNOTE.TXT for more details:

FrancisStokes 55 minutes ago [-]

Thanks! When it comes to performance I can understand choosing a more low level approach. construct-js trades performance for declarativeness (though I suspsect that I can do a lot of optimising under the hood).
As for DEFLATE, I'm working on an automatic bit level structure at the moment which would allow for unaligned structures. Should be in the lib in a couple of days.
Javascript  datastructures  binary 
5 days ago
Myspace lost all the music its users uploaded between 2003 and 2015 / Boing Boing
It's been a year since the music links on Myspace stopped working; at first the company insisted that they were working on it, but now they've admitted that all those files are lost: "As a result of a server migration project, any photos, videos, and audio files you uploaded more than three years ago may no longer be available on or from Myspace. We apologize for the inconvenience and suggest that you retain your back up copies. If you would like more information, please contact our Data Protection Officer, Dr. Jana Jentzsch at"

Yeah, apparently they don't have a backup.
cloud  outage  myspace 
5 days ago
Managed DNS Provider | Cloud DNS Service | Cloudflare
Fast & Secure Managed DNS Services

Powering over 39% of managed DNS domains, Cloudflare runs one of the largest authoritative DNS networks in the world. With an average of a few milliseconds query speed, we have the fastest performance of any managed DNS provider. DNS updates happen within seconds, making DNS propagation delays a thing of the past.
5 days ago
Meanwhile, on my end... - The Tao of Mac
There is a little bit of magic involved here–you need to know how the Objective-C bridge name mangling works, but I’ve had plenty of experience with the Python bridge, so it all boils down to a bit of experience and educated trial and error.

But it works fine for HEIC, DNG and CRW (Canon RAW), and I’ll be rewriting my Automator workflows to incorporate it as time permits.
EXIF  HEIC  DNG  Javascript 
5 days ago
Should you add a new APFS Container or Volume? – The Eclectic Light Company
APFS Volumes are quite different, though. Although you can set minimum and maximum sizes, known as Reserve and Quota respectively, they share free space. So a 1 TB disk can be divided up into three or four (or more) APFS Volumes each of 1 TB nominal size. Of course free space on that disk can’t be imagined out of nowhere: if one of those Volumes already contains 0.9 TB, the total free space available to all Volumes is only 0.1 TB.

Creating a new Volume on an existing APFS disk is simplicity itself, very quick, and non-destructive. Select the Container on the disk where you want the Volume to be, and click on the + tool above, labelled Volume. You’ll then be prompted to supply a name and select which of the APFS formats you want it to use.
5 days ago
The data brokers quietly buying and selling your personal information
Thanks to a new Vermont law requiring companies that buy and sell third-party personal data to register with the Secretary of State, we’ve been able to assemble a list of 121 data brokers operating in the U.S. It’s a rare, rough glimpse into a bustling economy that operates largely in the shadows, and often with few rules.
privacy  data 
5 days ago
The Slack Bubble and the reality of  the Professional Instant Messaging Market
“First of all, it is no longer relevant to consider separately the consumer and the business market. Inevitably, the two will merge. In your question, you mentioned email. Last year, a user had on average 1.7 email accounts; this year, the ratio will be 1.9, simply due to the increasing combination of professional and private use. Another example is the video call segment: look at your list of recent conversations on Skype, Zoom, or Hangout. They mix personal and professional activity. The line between private and personal use of these tools is more blurred than ever.
InstantMessaging  email  Skype  Slack 
5 days ago
Diagnosing diagnostics and Apple Hardware Test – The Eclectic Light Company
If you want to check your Mac’s hardware out, the first step is to run Diagnostics or Apple Hardware Test, which are detailed here. When you follow those instructions – disconnect all non-essential peripherals, connect keyboard and mouse/trackpad via USB, hold the D key and start up – eventually you should see the diagnostic test on offer. But what if you don’t?
MacOS  sysadmin 
5 days ago
Re-installing macOS: clean or what? – The Eclectic Light Company
It’s the last port of call. You’ve tried everything else, including installing the latest Combo updater package, and nothing has fixed your problem. There’s only one thing for it, to re-install macOS.

Should that be an install, re-install, or clean (re-)install? Just what is the difference now, and which should you do?
MacOS  sysadmin 
5 days ago
Getting started with the A1 instance | AWS Compute Blog
If you’re using open source software, everything you rely on most likely works on Arm systems today, and over the coming months we’ll be working on increasing the support and improving performance of software running on the A1 instances. If you have an open source based web-tier or containerized application, give the A1 instances a try and let us know what you think. If you run into any issues please don’t hesitate to get in touch at , via the AWS Compute Forum, or reach out through your usual AWS support contacts, we love customer’s feedback.
AWS  Arm 
6 days ago
Compressive Sensing Resources |
Compressive Sensing Resources
The dogma of signal processing maintains that a signal must be sampled at a rate at least twice its highest frequency in order to be represented without error. However, in practice, we often compress the data soon after sensing, trading off signal representation complexity (bits) for some error (consider JPEG image compression in digital cameras, for example). Clearly, this is wasteful of valuable sensing resources. Over the past few years, a new theory of "compressive sensing" has begun to emerge, in which the signal is sampled (and simultaneously compressed) at a greatly reduced rate. As the compressive sensing research community continues to expand rapidly, it behooves us to heed Shannon's advice.

Compressive sensing is also referred to in the literature by the terms: compressed sensing, compressive sampling, and sketching/heavy-hitters. To post new links or correct existing links, please email
compressedsensing  sensing  reference  resources 
6 days ago
Table of Contents - Rust Cookbook
Cookin' with Rust

This Rust Cookbook is a collection of simple examples that demonstrate good practices to accomplish common programming tasks, using the crates of the Rust ecosystem.

Read more about Rust Cookbook, including tips for how to read the book, how to use the examples, and notes on conventions.
rust  cookbook 
6 days ago
Adobe Lightroom for iOS Review: Industrial-strength image-editing tools on your iPhone or iPad - CNET
Bottom line
The mobile version of Lightroom CC started off as a pretty bare satellite editor with the desktop version of Lightroom as the planetary center. It has since matured into a full complement to Lightroom CC (and to a lesser extent Lightroom Classic) as an organizer and editor -- and can stand on its own, if necessary.

Android and iOS. Download for iOS and Android. Through a partnership between Adobe and, you can save 15 percent on a Creative Cloud subscription for the first year.
Competitive products
Affinity Photo for iPad. This app is more of a competitor to Photoshop than Lightroom, as it doesn't have a library component. However, it's an extremely sophisticated image editor that can also merge images for HDR and panorama photos.
Snapseed. One of the first popular mobile photo editors for iOS, Snapseed includes a huge array of adjustment tools.
RAW Power. This app uses the Photos library as the backbone for its library, and includes sophisticated controls for editing raw images (and other formats).
Lightroom  CC  iOS  review 
6 days ago
7 Commandments for Event-Driven Architecture | RJ Zaworski
Event-driven architecture is practically a requirement of the modern web stack. Coordinating microservices–nevermind serverless functions or an IOT mesh–via a stream of events minimizes interdependence and puts a clear vocabulary around service-to-service interactions. Systems get the resilience and scalability of minimally-dependent parts. Developers get a way to break big, complex problems down into reasonably isolated parts.

But with more moving parts and looser contracts between them, those nice, simple services tend to mean more complexity for the system as a whole. As you’re laying the foundation for an event-driven architecture, following a few rules can help keep things manageable.
architecture  microservices  events 
6 days ago
Fuchsia Friday: The structure of Google's Lego-like modular OS, explained - 9to5Google
This week in Fuchsia Friday we look at the bigger picture. Google is making Fuchsia fundamentally different from systems of the past, especially learning from the challenges the company has encountered with Android. But now that we’ve taken a few weeks to describe some of its more important elements in detail, let’s take a step back and look at the big picture. Let’s check out the building blocks and terminology that Google uses to describe Fuchsia’s “modular” design.
Google  OS  Fuchsia 
6 days ago
Cyber Attack on VFEmail Evaporates 18 Years of Data! | - The Home of Storage Switzerland
In what may be one of the most devastating attacks in recent history, email provider VFEmail is working through the aftermath of a massive cyberattack and is indicating that 18 years’ worth of its user’s email data is lost forever. The attack seems to have reformatted the volumes in all of VFEmail’s US servers, including the volumes of backup servers. The virtual machines used by VFEmail were different operating systems, and the company claims that authentication into each machine was different.
cloud  outage  email 
6 days ago
JSweet: a transpiler to write JavaScript programs in Java
A transpiler from Java to TypeScript/JavaScript
1000+ well-typed JavaScript libraries available from Java

Version 2.0.0
opensource  java  javascript 
7 days ago
Plugable USBC-NVME USB 3.1 Gen 2 NVMe SSD Enclosure Review | - Storage Reviews
While there are a ton of options when it comes to portable storage, there are very few when it comes to portable NVMe. Plugable is offering a good compromise that allows users to take full advantage of NVMe SSD in a portable manner and allows users to choose their drive. The able USBC-NVME Enclosure is small and stylish fitting various M.2 form factors and leverages USB Type C interface.
nvme  ssd  storage 
7 days ago
Choose Technology Suppliers Carefully – Perspectives
However, the second thing I’ve learned from watching Oracle/customer relationships over the years is there really are limits to what customers will put up with and when hit, action is taken.  Eventually, a customer will focus on the “Oracle Problem” and do the work to leave the unhealthy relationship. When they do, the sense of relief in the room is absolutely palpable. I’m super happy to have been part of many of these migrations in my near-decade on each of IBM DB2 and Microsoft SQL Server. It’s fun when helping to make your product work for a customer not only helps your company but also helps the customer save money, get a better product, and no longer have to manage an unhealthy supplier relationship.
Oracle  AWS 
7 days ago
Gierad Laput | Synthetic Sensors
We explore the notion of general-purpose sensing, wherein a single, highly capable sensor can indirectly monitor a large context, without direct instrumentation of objects. Further, through what we call Synthetic Sensors, we can virtualize raw sensor data into actionable feeds, whilst simultaneously mitigating immediate privacy issues. We use a series of structured, formative studies to inform the development of new sensor hardware and accompanying information architecture. We deployed our system across many months and environments, the results of which show the versatility, accuracy and potential of this approach.
sensor  sensors  iot  measurement  analytics 
7 days ago
Cloud computing simplified: a Berkeley view on serverless computing | the morning paper
Ten years ago Berkeley released the ‘Berkeley view of cloud computing’ paper, predicting that cloud use would accelerate. Today’s paper choice is billed as its logical successor: it predicts that the use of serverless computing will accelerate. More precisely:

… we predict that serverless computing will grow to dominate the future of cloud computing.

The acknowledgements thank reviewers from Google, Amazon, and Microsoft among others, so it’s reasonable to assume the major cloud providers have had at least some input to the views presented here. The discussion is quite high level, and at points it has the feel of a PR piece for Berkeley (there’s even a cute collective author email address: serverlessview at, but there’s enough of interest in its 35 pages for us to get our teeth into…
serverless  analysis  paper 
7 days ago
GitHub - chanind/curve-matcher: Curve matching using Fréchet distance and Procrustes analysis in JS
A Javascript library for doing curve matching with Fréchet distance and Procrustes analysis.
geometry  gis  gps  Javascript 
7 days ago
Hardware innovation for data growth challenges at cloud-scale | Blog | Microsoft Azure
We are open sourcing Project Zipline compression algorithms, hardware design specifications, and Verilog source code for register transfer language (RTL) with initial content available today and more coming soon. This contribution will provide collateral for integration into a variety of silicon components (e.g. edge devices, networking, offload accelerators etc.) across the industry for this new high-performance compression standard. Contributing RTL at this level of detail as open source to OCP is industry leading. It sets a new precedent for driving frictionless collaboration in the OCP ecosystem for new technologies and opening the doors for hardware innovation at the silicon level. Over time, we anticipate Project Zipline compression technology will make its way into several market segments and usage models such as network data processing, smart SSDs, archival systems, cloud appliances, general purpose microprocessor, IoT, and edge devices.

Project Zipline is a cutting-edge compression technology optimized for a large variety of datasets, and our release of RTL allows hardware vendors to use the reference design to produce hardware chips to allow the highest compression, lowest cost, and lowest power out of the algorithm. It's available to the OCP ecosystem, so they can contribute to it, and create further benefit for the entire ecosystem, including Azure and our customers.
zip  compression  opensource  hardware  software 
7 days ago
How to choose the best GPS for your iPad - iPad Pilot News
Our picks

Which one is best for pilots? None of these GPSs is really a bad choice, but we’ll offer two picks. For a good performer at a good price, the Dual XGPS150A is hard to beat. At just $99.95, it offers good battery life, reliable performance and a handy dash mount. It has been one of the best-selling models for years and gets good reviews from pilots.

For a deluxe model, or for pilots who fly with multiple devices in the cockpit, we like the new Bad Elf Pro+. At $249.99 it isn’t cheap, but it’s very well made and has a number of great features: an incredible 35-hour battery life, handy built-in screen, altimeter and connection to multiple devices.

You can see the entire selection of iPad GPSs at Sporty’s.
iPad  GPS 
7 days ago
The Bitter Lesson
One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning.

The second general point to be learned from the bitter lesson is that the actual contents of minds are tremendously, irredeemably complex; we should stop trying to find simple ways to think about the contents of minds, such as simple ways to think about space, objects, multiple agents, or symmetries. All these are part of the arbitrary, intrinsically-complex, outside world. They are not what should be built in, as their complexity is endless; instead we should build in only the meta-methods that can find and capture this arbitrary complexity. Essential to these methods is that they can find good approximations, but the search for them should be by our methods, not by us. We want AI agents that can discover like we can, not which contain what we have discovered. Building in our discoveries only makes it harder to see how the discovering process can be done.
AI  ML  trend 
8 days ago
Apple’s iCloud is the latest service to suffer an outage this week – BGR
By my count, there are twenty problematic services at the time of writing, with all of the issues having begun at 11:00 a.m. this morning. Apple hasn’t provided any additional details beyond which services are struggling, mostly using a single phrase to explain every issue: “Users may be unable to access this service.”
cloud  outage  apple 
8 days ago
The Math That Tells Cells What They Are | Quanta Magazine
That prompted a group at Princeton University, led by the biophysicists Thomas Gregor and William Bialek, to suspect something else: that the cells could instead get all the information they needed to define the positions of pair-rule stripes from the expression levels of the gap genes alone, even though those are not periodic and therefore not an obvious source for such precise instructions.

And that’s just what they found.

Over the course of 12 years, they measured morphogen and gap-gene protein concentrations, cell by cell, from one embryo to the next, to determine how all four gap genes were most likely to be expressed at every position along the head-to-tail axis. From those probability distributions, they built a “dictionary,” or decoder — an explicit map that could spit out a probabilistic estimate of a cell’s position based on its gap-gene protein concentration levels.

Around five years ago, the researchers determined this mapping by assuming it worked like what’s known as an optimal Bayesian decoder (that is, the decoder used Bayes’ rule for inferring the likelihood of an event from prior conditional probabilities). The Bayesian framework allowed them to flip the “unknowns,” the conditions of probability: Their measurements of gap gene expression, given position, could be used to generate a “best guess” of position, given only gap gene expression.
bioinformatics  geolocation  bayesian 
9 days ago
A One-Page Guide to Fixing Radiated Emissions
Radiated emissions are a measure of how much electronic noise an electronic product emits into its surroundings. The government sets limits on this noise with the goal of regulating the limited electronmagnetic spectrum available to legitimate transmitters - i.e., wireless communications.

That’s a delightful description, but not a particularly useful. I’m willing to bet that, if you’re reading this page, “radiated emissions” likely means “a test your design is failing”.
radiation  EMI 
9 days ago
Roll Your Own Analytics - PC Maffey
The Stack

I wanted to set this up quickly, so I decided on the most easily accessible tools for my current go-to stack.* You could swap out any of these layers for your preferred tech. If you're expecting higher usage, you'll almost certainly want a more robust data store.*
Here's how it works:
Log a session on the client (React, state + context API)
When the session ends...
Call a lambda function (AWS Lambda via Netlify)
Store the data in Google Sheets (Google Sheets, Google App Scripts)
monitoring  measurement  analytics 
9 days ago
Introducing Firefox Send, Providing Free File Transfers while Keeping your Personal Information Private - The Mozilla Blog
Share large files & navigate with ease
Send also makes it simple to share large file sizes – perfect for sharing professional design files  or collaborating on a presentation with co-workers. With Send you can share file sizes up to 1GB quickly. To send files up to 2.5GB, sign up for a free Firefox account.

Send makes it easy for your recipient, too. No hoops to jump through. They simply receive a link to click and download the file. They don’t need to have a Firefox account to access your file. Overall, this makes the sharing experience seamless for both parties, and as quick as sending an email.
firefox  mozilla  tools  files 
9 days ago
Why Every Influencer Has a Discord - The Atlantic
Discord is a real-time chat platform that was founded four years ago as a way to make it easier for gamers to communicate. But over the past year, it has outgrown its origin story and become the default place where influencers, YouTubers, Instagram meme accounts, and anyone with an audience can connect with their community.
Discord  chat  socialmedia  trend  decentralized 
9 days ago
Discord (software) - Wikipedia
Discord is a proprietary freeware VoIP application and digital distribution platform designed for video gaming communities, that specializes in text, image, video and audio communication between users in a chat channel. Discord runs on Windows, macOS, Android, iOS, Linux, and in web browsers. As of December 2018, there are over 200 million unique users of the software.
discord  chat  decentralized  socialmedia 
9 days ago
Radicle 🌱
The Radicle stack builds on IPFS, and works without a centralized server. No hassle with setting up your own hosting, and no need to trust companies with your data.
p2p  git  github  IDE  decentralized 
9 days ago
Channable - How we made Haskell search strings as fast as Rust
We set out to speed up string searching, a common operation for Channable. We investigated several Aho–Corasick implementations, and concluded that our best bet was to build a fast Haskell implementation, which we are open-sourcing today. Though Haskell is not the best tool for doing low-level optimization, it is definitely possible. In the end our implementation came out faster than any of the others we had tried before.

Finding the right way to instruct GHC to generate fast code remains a challenge. Although the end result is code that is clear and fast, the journey to get there was quite the adventure. Relying so much on optimizations to ensure that very high level code compiles to the right low-level code is fragile. Case in point: after upgrading our Stackage snapshot from LTS 10 with GHC 8.2.2 to LTS 13 with GHC 8.6.3, we saw the running time of our benchmark double. After some investigation we got the times back down, but this involved a subtle combination of code changes and inlining pragmas. Still, our journey was definitely worth the trouble, for in the end we can service users with even faster previews and a more responsive tool.
haskell  optimization  rust  performance  search 
9 days ago
Iodide: an experimental tool for scientific communication and exploration on the web – Mozilla Hacks – the Web developer blog
Iodide is a tool designed to give scientists a familiar workflow for creating great-looking interactive documents using the full power of the web platform. To accomplish that, we give you a “report” — basically a web page that you can fill in with your content — and some tools for iteratively exploring data and modifying your report to create something you’re ready to share. Once you’re ready, you can send a link directly to your finalized report. If your colleagues and collaborators want to review your code and learn from it, they can drop back to an exploration mode in one click. If they want to experiment with the code and use it as the basis of their own work, with one more click they can fork it and start working on their own version.
notebook  Iodide  jupyter 
9 days ago
A Beginner's Guide to Attention Mechanisms and Memory Networks | Skymind
Attention mechanisms in neural networks serve to orient perception as well as memory access (you might even say perception is just a very short-term subset of all memory). Attention filters the perceptions that can be stored in memory, and filters them again on a second pass when they are to be retrieved from memory. Attention can be aimed at the present and the past.

Attention matters because it has been shown to produce state-of-the-art results in machine translation and other natural language processing tasks, when combined with neural word embeddings, and is one component of breakthrough algorithms such as Transformer and BERT, which is setting new records in accuracy in NLP. So attention is part of our best effort to date to create real natural-language understanding in machines. If that succeeds, it will have an enormous impact on society and almost every form of business.
10 days ago
Google's Gmail and Drive suffer global outages | Technology | The Guardian
Google has been hit by outages in a host of countries around the world, with users reporting issues with Gmail, Google Drive, Hangouts and Google Maps for several hours.

Various websites that track Gmail problems and outages, showed a spike in users reporting problems with the email service from about 1pm AEDT (2am GMT).

In a statement on their service website, Google said: “We’re investigating reports of an issue with Gmail. We will provide more information shortly. The affected users are able to access Gmail, but are seeing error messages, high latency, and/or other unexpected behaviour.”

Google reported that the issues had been resolved at 5.30pm AEDT (6.30am GMT).
cloud  outage  google 
10 days ago
MIT Spin off Supercomputing Startup Conduit Launches from Stealth
Founded in 2017 by MIT mechatronics and information theory graduate Ryan Robinson, Conduit wants to compete with behemoths like Amazon web services and Microsoft Azure. Currently, it has a staff of eight full-time employees.

The startup is currently working with mid-sized pharma companies in the Boston area but in the future plans to add clients in fintech, supply chain logistics and quantum computing.
decentralized  Conduit 
10 days ago
Conduit - Serious Computing for the Future
A decentralized Cloud computing service
Conduit is an MIT spinoff working to solve the computing advances we face as technology and AI evolve. We are looking to revolutionize the way companies obtain and utilize computing power by gathering the world’s extra computing resources to provide the most powerful cloud solution.
decentralized  AWS 
10 days ago
Identity by Any Other Name - ACM Queue

It used to be that we focused on one application running on one computer accessing one SQL database. While we may have had application-based identifiers (e.g., Social Security numbers), the underlying system was based on values in cells. Relational algebra related values to other values.

As systems cleave apart for scale, cleave apart to provide management or trust boundaries, or cleave together to integrate solutions, identifiers and identity form the glue that binds solutions. Identities also formalize the separation of disparate and distrusting solutions. Cleaving apart or cleaving together requires identities.

When we bind work together with identities, the interesting tension is, "What constitutes the identity?" What precisely is identified by a king-sized nonsmoking room? Where did we deliver the message that was guaranteed to be delivered?

New emerging systems and protocols both tighten and loosen our notions of identity, and that's good! They make it easier to get stuff done. REST, IoT, big data, and machine learning all revolve around notions of identity that are deliberately kept flexible and sometimes ambiguous. Notions of identity underlie our basic mechanisms of distributed systems, including interchangeability, idempotence, and immutability.

Finally, don't be too picky about calling this identity. We see identity as names, keys, pointers, handles, IDs, numbers, identifiers, UUIDs, GUIDs, document IDs, UPCs, ASINs, employee numbers, file names, Social Security numbers, and much more.

Truly, identity by any other name does smell as sweet
identity  distributed  decentralized  archiving  cs 
11 days ago
Decentralized Web Summit 2018: Global Visions / Working Code
About People Science Fair Press Kit FAQ Videos
Decentralized Web Summit 2018
Global Visions / Working Code
July 31 – August 2, 2018
→ Videos from the Summit
decentralized  dweb 
11 days ago
Web inventor urges users to seek 'complete control' of data
Berners-Lee, who last year launched a development platform called "Solid" aimed at giving users control of their data, described a frightening future if we do not rise to the challenge of privacy protection.
decentralized  solid 
11 days ago
Tutorial: Merkle Trees and the IPFS DAG · Decentralized Web Primer
Tutorial: Merkle Trees and the IPFS DAG


These Lessons introduce the following concepts:

Cryptographic Hashes and Content Addressability
Authenticated Graphs
Turning Files into Trees
Turning any Data into Trees
Publishing hashes on the DHT
Getting data from the Peer to Peer Network


Lesson: Turn a file into a tree of hashes
Lesson: Create a cryptographic hash
Lesson: Build a tree of data in IPFS using cryptographic hashes to link the pieces (a Merkle DAG)
Lesson: Explore the types of software that use hash trees to track data
IPFS  MerkleTree  blockchain 
12 days ago
Kdenlive | Libre Video Editor
Kdenlive is an open source video editor. The project was started around 2003. Kdenlive is built on Qt and the KDE Frameworks libraries. Most of the video processing is done by the MLT Framework, which relies on many other open source projects like FFmpeg, frei0r, movit, ladspa, sox, etc…
Our software was designed to answer most needs, from basic video editing to professional work.
Kdenlive is developed by a small team and new contributors are welcome.
video  editor  linux 
12 days ago
Data Terra Nemo

Data Terra Nemo is a technical conference for hackers and computer scientists focused on distributed and decentralized systems.
decentralized  conference  Berlin  dweb  p2p  data 
12 days ago
GitHub - rurban/smhasher: Improved fork of
So the fastest hash functions on x86_64 without quality problems are:

falkhash (macho64 and elf64 nasm only, with HW AES extension)
Metro (but not 64crc yet, WIP)
FarmHash (not portable, too machine specific: 64 vs 32bit, old gcc, ...)
City (deprecated)
mum (machine specific, mum: different results on 32/64-bit archs)
algorithms  hashing  performance 
12 days ago
Basecamp 2 and Basecamp 3 search outage report - Signal v. Noise
From 4:30am GMT March 7 / 10:30pm Central March 6 until 1:02pm GMT / 7:02am Central, Basecamp 2 and the search feature in Basecamp 3 were mostly offline due to a catastrophic network failure with our cloud provider. Both our primary network link, our backup network link, and several additional ad-hoc network links between critical services needed to run Basecamp 2 were forced offline, as the cloud provider sought to deal with underlying network problems they were having.

Both Basecamp 2 and the search feature in Basecamp 3 are now fully back online.
cloud  outage 
12 days ago
restic · Backups done right!
restic is a program that does backups right. The design goals are:

Easy: Doing backups should be a frictionless process, otherwise you are tempted to skip it. Restic should be easy to configure and use, so that in the unlikely event of a data loss you can just restore it. Likewise, restoring data should not be complicated.

Fast: Backing up your data with restic should only be limited by your network or hard disk bandwidth so that you can backup your files every day. Nobody does backups if it takes too much time. Restoring backups should only transfer data that is needed for the files that are to be restored, so that this process is also fast.

Verifiable: Much more important than backup is restore, so restic enables you to easily verify that all data can be restored.

Secure: Restic uses cryptography to guarantee confidentiality and integrity of your data. The location where the backup data is stored is assumed to be an untrusted environment (e.g. a shared space where others like system administrators are able to access your backups). Restic is built to secure your data against such attackers, by encrypting it with AES-256 in counter mode and authenticating it using Poly1305-AES.

Efficient: With the growth of data, additional snapshots should only take the storage of the actual increment. Even more, duplicate data should be de-duplicated before it is actually written to the storage backend to save precious backup space.

Free: restic is free software and licensed under the BSD 2-Clause License and actively developed on GitHub.
backup  encryption  go  opensource 
13 days ago
Append-only backups with restic and rclone
One issue with most backup solutions is that an attacker controlling the local system can also wipe its old backups. To prevent this the backup must permit append-only backups (also called add-only backups). Restic is a sophisticated backup tool which is easy to use, supports encryption and many backends to store the data. In combination with rclone it can be used to support append-only backups. The goal of this guide is to convert regular restic backups via SFTP to support append-only backups.
backup  sysadmin 
13 days ago
« earlier      
per page:    204080120160

Copy this bookmark:

to read