https://blog.stites.io/posts/ Recent content in Posts on Sam Stites Hugo -- gohugo.io en-US Copyright © 2023, Sam Stites. Fri, 02 Nov 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-11-02-hasktorch-release/ Fri, 02 Nov 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-11-02-hasktorch-release/ Long story short: After a few prototypes and feedback, I’m happy to announce that hasktorch-0.0.1.0 is now officially on Haskage! The API documentation can be found here. The rest of this post introduces the library and shows you how to build a fully-connected XOR predictor. In a followup post, I’ll talk a bit about internals, go over some caveats, talk about what future development entails, and hopefully get you all excited about contributing : ) https://blog.stites.io/posts/2018-08-24-hyperparameters/ Fri, 24 Aug 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-08-24-hyperparameters/ There are really two classes of these: optimizer hyperparameters (learning rate, number epochs, minibatch size, optimizer), and model hyperparameters (number and size of layers). From what I’ve seen optimizer hyperparameters are the parameters where there will one day be algorithm solutions to removing them from your model. Things like learning rate finders and cyclical learning rates (Cyclical Learning Rates for Training Neural Networks, Smith), adaptive batch sizes (AdaBatch, Devarakonda, et al), and convergence properties (really just borrowing from reinforcement learning literature, but Super-Convergence - Smith, Topin seems to include a proof for convergence) are lines of research with varying sucess at removing these. https://blog.stites.io/posts/2018-08-23-gap-layer/ Thu, 23 Aug 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-08-23-gap-layer/ On Global Average Pooling (GAP) layers, From Network In Network, Lin, et al: In this paper, we propose another strategy called global average pooling to replace the traditional fully connected layers in CNN. The idea is to generate one feature map for each corresponding category of the classification task in the last mlpconv layer. Instead of adding fully connected layers on top of the feature maps, we take the average of each feature map, and the resulting vector is fed directly into the softmax layer. https://blog.stites.io/posts/2018-08-22-lstm-notes/ Wed, 22 Aug 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-08-22-lstm-notes/ Scenario: “Say we have a network that watches TV and classifies if an object is a wolf, dog, bear, goldfish, or an unknown thing. With a simple feed forward, convolutional network we would look at snapshots of frames and make a prediction for each animal. This is pretty simplisitic.” Because we are working with an ordered sequence over time, though, we can feed in hints of what the network has seen before. https://blog.stites.io/posts/2018-08-21-more-dl-notes/ Tue, 21 Aug 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-08-21-more-dl-notes/ Some quick reminders: On Dropout: Don’t forget to remove dropout on validation! We don’t want to overfit when training, but on inference we would like to use the full power of our model. Autoencoders: Currently, these aren’t good at lossless compression (obvious, since they output probabilistic data structures), and aren’t so good at video compression (which I specify because you could probably allow for some loss-of-information). That said, if I were to try solving that problem, I would mix autoencoders with neural architecture search – with model size + compression size as the output (+ accuracy) as the bandit reward. https://blog.stites.io/posts/2018-08-15-different-ways-to-slice/ Wed, 15 Aug 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-08-15-different-ways-to-slice/ Some guidelines for how to slice a network when transfer learning. If the dataset is small and the new training data is similar to the original network’s data: Slice off only the last layer (not the entire head). If the dataset is small and the new training data is different to the original network’s data: Slice off the entire head and replace it with ONE fully-connected layer. If the dataset is large (up to your discretion) and the new training data is similar to the original data: Slice off the entire head and, matching the architecture, replace it with a new fully-connected network. https://blog.stites.io/posts/2018-08-14-data-augmentation-is-a-hack/ Tue, 14 Aug 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-08-14-data-augmentation-is-a-hack/ Ideally, models are invariant to translation, rotation, and scaling. You would hope that your learning model identifies you both in close-up photos, as well as in a group shots (even if one is just a photoshopped version of the other). That said, convolutional layers are only translation-invariant – and only somewhat translation-invariant at that! This is where data augmentation comes in – making sure that our model is trained on data which would make models robust to translation, rotation, and scaling (hopefully resulting in invariance). https://blog.stites.io/posts/2018-08-02-git-find-my-bugs/ Thu, 02 Aug 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-08-02-git-find-my-bugs/ Two different git commands today: $ git rebase -i --exec "make CFLAGS='-Werror' e7509a8c03 Runs a linear pass through every commit between e7509a8c03 and HEAD, building the project and stopping the rebase if the build fails. This allows you to fix the commit and continue. You can, obviously, put any command in the --exec flag. Good for hard-moding a small segment of git commits. $ git bisect start HEAD e7509a8c03 -- # HEAD is bad, e7509a8c03 is good $ git bisect run make # "make" builds the app $ git bisect reset # quit the bisect session More traditional: git’s binary search command to find a bug between e7509a8c03 and HEAD. https://blog.stites.io/posts/2018-07-18-nixing/ Wed, 18 Jul 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-07-18-nixing/ Reviewing Gabriel439/haskell-nix to prepare myself for a travelling development workflow. Here are some short notes: nix-shell requires an env attribute. build.nix files should return a set. build.nix files, in project0, should specify projects to build in with project0 = pkgs.haskellPackages.callPackage ./default.nix {} elements In the above case, you enter a nix-shell with nix-shell --attr project0.env release2.nix you can default this with a shell.nix of (import ./release2.nix).project0.env https://blog.stites.io/posts/2018-07-11-rl-notes/ Wed, 11 Jul 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-07-11-rl-notes/ Things to remember: discounted return is with respect to the future even though, pragmatically, this is usually seen as an eligibility trace. This is because of the Markov Property. $\gamma = 0$ implies a myopic agent – one that only cares about immediate reward. $\gamma = 1$ implies a farsighted agent – one that cares about optimizing future reward. $0 < \gamma < 1$ implies that an agent cares about the future to varying degrees. https://blog.stites.io/posts/2018-07-10-debugging-ml/ Tue, 10 Jul 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-07-10-debugging-ml/ There are various tips when debugging machine learning algorithms, which recently came up in the Torch gitter channel. Documenting some of these words of wisdom: Write unit test with fixed seeds and prefilled tensors for a step or two You may also get to rollout half the epoch or episode (depending on your domain, episodes only if you can adjust the environment) and then unit test some more. In general stepping through your code with a debugger comes first, unit tests are just to codify that. https://blog.stites.io/posts/2018-06-12-deep-reinforcement-learning-that-matters/ Tue, 12 Jun 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-06-12-deep-reinforcement-learning-that-matters/ A quick write up of some notes on Deep Reinforcement Learning that Matters that I took on the plane. So the paper itself focuses on Model-Free Policy Gradient methods in continuous environments and is an investigation into how reproducing papers in the Deep Reinforcement Learning space is notoriously difficult. The authors discuss various failure cases that any researcher will be privy to when trying to implement work, and the shortcomings of the majority of authors who follow standard publication practices. https://blog.stites.io/posts/2018-01-30-geforce-quadros-and-teslas/ Tue, 30 Jan 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-01-30-geforce-quadros-and-teslas/ I’ve been thinking of expanding my deep learning rig, repurposing one of my 1080Ti cards for mining (on a PCIe 8x lane), while bumping up that 16-lane slot to a more “professional” deep learning card. In the GeForce series, from what I understand, a 1080 Ti is already a good enough bang-for-your-buck and a Titan is only an incremental improvement. My coworker has tauted having a Quadro series (which might be less expensive than a Tesla), but ultimately is better suited for 3d-acceleration than for deep learning. https://blog.stites.io/posts/2018-01-25-month-percentiles/ Thu, 25 Jan 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-01-25-month-percentiles/ Ways to handle the day and month features in a timeseries dataset: Treat all months as constant (ie: “all months are 30 days”) – good but we loose information and February becomes an awkward month (also, leap years are not treated in a special fashion which might be important). Computationally efficient, semantically incorrect, losses information. I think I see this one the most “in the wild” and it always looks like a hack. https://blog.stites.io/posts/2018-01-23-structured-deep-learning/ Tue, 23 Jan 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-01-23-structured-deep-learning/ Going through more of fastai – Jeremy Howard points out that there is a huge gap between industry and academia where academia loves solving unsupervised learning problems and industry loves supervised ones (guess I’m lucky, working in semi-supervised learning). This is pretty easy to see since unsupervised learning is the task of understanding what you do not know, whereas supervised learning is just memorization. This point can be seen through Andrew Ng and Geoffery Hinton’s interview on the deeplearning. https://blog.stites.io/posts/2018-01-22-clipping-for-multiclass/ Mon, 22 Jan 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-01-22-clipping-for-multiclass/ For the past couple of days I’ve been writing a transfer-learning workflow for the dog breed identification kaggle competition. This was done solely in pytorch and torchvision, heavily referencing the beginner pytorch tutorials, and you can find them at stites/circus. This code, which doesn’t do anything fancier than a fastai notebook, was able to produce results that placed in the 50th-percentile of the competition. This isn’t enough to qualify for anything like a bronze medal on kaggle (there are ~1k participants), but as a tool to familiarize myself with a CV pipeline from scratch, I would say this is a success. https://blog.stites.io/posts/2018-01-12-sigmoid-activation/ Fri, 12 Jan 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-01-12-sigmoid-activation/ Use sigmoid activation for multilabel classification (not softmax!) in your output layer for multilabel classification and use binary crossentropy for your loss/cost function. Sigmoid, unlike softmax, doesn’t give you a probability distribution around the number of classes, and so doesn’t assume a univariate distribution. This is, essentially, using the same mechanics as logistic regression but doen’t normalize across the total output classes. Use a cutoff of 0.5 to determine if something is in- or out-of class. https://blog.stites.io/posts/2018-01-11-cyclical-learning-rates/ Thu, 11 Jan 2018 00:00:00 +0000 https://blog.stites.io/posts/2018-01-11-cyclical-learning-rates/ Cyclical Learning Rates for Training Neural Networks (github) Introduced via Jeremy Howard as one of the two techniques that gives him deep learning superpowers, cyclical learning rates are a departure from the manual decision making process when choosing a learning rate in deep learning. Currently either stepwise- or cosine-annealing is the usual method of learning rate decay, but these include decisions around manual hyperparameters which humans are not nessecarily good at making. https://blog.stites.io/posts/2017-12-27-activation-functions-and-non-linearity/ Wed, 27 Dec 2017 00:00:00 +0000 https://blog.stites.io/posts/2017-12-27-activation-functions-and-non-linearity/ if linear functions are \(s = W x\), then a 2-layer neural network function would look more like look more like: $$ s = W_2 max(0, W_1 x) $$ Where \(W_1\) has the dimensions of input and an intermediate dimension and \(W_2\) mapping from the intermediate dimension to our output dimension. Notice that if we remove the element-wise \(max\) function then we would be able to collapse \(W_2 W_1\) into a single matrix. https://blog.stites.io/posts/2017-12-27-hyperparameter-guidance/ Wed, 27 Dec 2017 00:00:00 +0000 https://blog.stites.io/posts/2017-12-27-hyperparameter-guidance/ When writing your next neural network you’ll need to go about selecting certain hyperparameters like number of layers, the size of each layer, how many convolutional layers, and how large your filter sizes should be. Theoretically, a single neural network with a single hidden layer is a universal approximator to any function (theory1, alternate explanation). Intuitively, as well, we would assume that the less complicated the model, the less likely we will overfit our training data. https://blog.stites.io/posts/2017-12-27-measuring-the-size-of-neural-networks/ Wed, 27 Dec 2017 00:00:00 +0000 https://blog.stites.io/posts/2017-12-27-measuring-the-size-of-neural-networks/ Neural networks are sized in one of two ways: the number of neurons, or the number of trainable parameters. When given the following two neural networks: We can either say that this 2-layer network has \(4 + 2 = 6\) neurons (don’t include the inputs), or that the network has \(3 * 4 + 4 * 2 = 20\) weights and \(4 + 2 = 6\) biases for a total of 26 learnable parameters. https://blog.stites.io/posts/2017-12-23-optimization-notes/ Sat, 23 Dec 2017 00:00:00 +0000 https://blog.stites.io/posts/2017-12-23-optimization-notes/ In Stanford’s CS231n first module titled: “Optimization: Stochastic Gradient Descent,” they describe how loss for multiclass SVM without regularization is described as: $$L_i = \sum_{j\neq y_i} \left[ \max(0, w_j^Tx_i - w_{y_i}^Tx_i + 1) \right]$$ For their three-dimensional example across (W). Breaking this down across each row of (W), we get: $$L_0 = \max(0, w_1^Tx_0 - w_0^Tx_0 + 1) + \max(0, w_2^Tx_0 - w_0^Tx_0 + 1)$$ $$L_1 = \max(0, w_0^Tx_1 - w_1^Tx_1 + 1) + \max(0, w_2^Tx_1 - w_1^Tx_1 + 1)$$ $$L_2 = \max(0, w_0^Tx_2 - w_2^Tx_2 + 1) + \max(0, w_1^Tx_2 - w_2^Tx_2 + 1)$$ $$L = (L_0 + L_1 + L_2) / 3 $$ https://blog.stites.io/posts/2017-10-31-maybe-py/ Tue, 31 Oct 2017 00:00:00 +0000 https://blog.stites.io/posts/2017-10-31-maybe-py/ I was getting a little frustrated with python’s try-catch style of programming and made a rough maybe type which seems to work well. It’s a little hacky, but you can fmap and bind, plus there are some fun little list helpers: https://blog.stites.io/posts/2017-10-09-bic-dic-cv/ Mon, 09 Oct 2017 00:00:00 +0000 https://blog.stites.io/posts/2017-10-09-bic-dic-cv/ Comparison of BIC, DIC, and CV: BIC: In information criterion like AIC, BIC, and DIC, the primary role is to come up with a criteria that will both reward high likelihood, but penalizes model complexity. In this way a model will both be a good fit for the data and will also generalize well beyond the data. One of BIC’s advantages is that it prefers simpler models than its predecessor, AIC. https://blog.stites.io/posts/2017-10-06-gotty/ Fri, 06 Oct 2017 00:00:00 +0000 https://blog.stites.io/posts/2017-10-06-gotty/ Check out gotty! For no-write access over http: $ gotty tmux new -A -s showoff $ ssh -t -R 9001:localhost:8080 $(whoami):my-box.com sudo socat tcp-listen:80,reuseaddr,fork tcp:localhost:9001 $ tmux new -A -s showoff For write-access over https with basic auth (after making tls-certs): $ openssl req -x509 -nodes -days 9999 -newkey rsa:2048 -keyout ~/.gotty.key -out ~/.gotty.crt $ gotty -w -c user:pass -t tmux new -A -s collab $ ssh -t -R 9001:localhost:8080 $(whoami):my-box. https://blog.stites.io/posts/2017-09-13-generous-travis/ Wed, 13 Sep 2017 00:00:00 +0000 https://blog.stites.io/posts/2017-09-13-generous-travis/ I was hitting an out-of-memory error in travis and found that adding sudo:required to a .travis.yml will switch your build to use their standard infrastructure (currently 3GB ram on 2 cores, according to the FAQ). Without this, travis uses containers and hitting upper limits for any build becomes more of a problem. https://blog.stites.io/posts/gtd/2017-09-10-gtd-week1/ Sun, 10 Sep 2017 00:00:00 +0000 https://blog.stites.io/posts/gtd/2017-09-10-gtd-week1/ Starting to document some workflow and do some review around my latest attempt at GTD with remember the milk. This might sound silly writing it out – hopefully no one ever reads this. Okay things that are working so far: all current “workflows” are encompassed (reading things in pocket, queuing up project wishlists, managing action items in multiple projects). Plus it’s not super painful. I’m able to replicate “habits” just like in the Fabulous app (although I don’t have some kind of guide to tell me about cute little embellishments), also it’s way less ostentatious than Fabulous. https://blog.stites.io/posts/2017-08-31-deriveanynewtypereader-considered-harmful/ Thu, 31 Aug 2017 00:00:00 +0000 https://blog.stites.io/posts/2017-08-31-deriveanynewtypereader-considered-harmful/ So for the past couple of days I’ve been chasing down what I thought was a space leak introduced by monadic while loops. It’s a reasonable thing to think because reinforcement learning in haskell requires nested monadic loops: one outer loop to iterate through all episodes, another nested loop to travel through all steps in each episode. A pattern which has been documented by others as potentially leaky but which, never-the-less, is still sometimes useful*. https://blog.stites.io/posts/2017-07-16-least-interesting-data-caches/ Sun, 16 Jul 2017 00:00:00 +0000 https://blog.stites.io/posts/2017-07-16-least-interesting-data-caches/ Techniques like experience replay, sampled traces, and what I am expecting to read about in [the sample-efficent Reactor architecture][reactor], all seem to be targeting the idea that we want to efficiently reuse experience for efficent and stable learning. Also of note is that DQN and DRQN architectures use a dedicated memory buffer which is randomly sampled for faster learning. So a quick thought that came to me: can we do better than random sampling if we are occasionally antifragile? https://blog.stites.io/posts/2017-07-16-policy-vs-value-functions/ Sun, 16 Jul 2017 00:00:00 +0000 https://blog.stites.io/posts/2017-07-16-policy-vs-value-functions/ Usually, policy and value functions work best, but most of the time finding a policy is preferred over finding a value function. Of course this it dependent on your use case – it might not be possible to find a policy as easily as a value function, or finding a policy might over-simplify the situation. Part of what makes policies nice is the fact that they can be represented in a more compact manner. https://blog.stites.io/posts/2017-05-19-tmux-time/ Fri, 19 May 2017 00:00:00 +0000 https://blog.stites.io/posts/2017-05-19-tmux-time/ Cool beans! <Prefix>-t displays the time in tmux! Since moving to xmonad on FreeBSD, little things have become a lot harder and, much in the same way with haskell, I have that feeling of, “before I felt like an expert and everything broke, now I feel like a beginner and everything works perfectly.” My time has also been diverted to towards the huge project of researching Machine Intelligence and implementing a full system here at Sentenai, but I think it’s important to write about the little, possibly silly, technical things as well to keep progress moving. https://blog.stites.io/posts/2017-05-06-multi-step-bootstrapping/ Sat, 06 May 2017 00:00:00 +0000 https://blog.stites.io/posts/2017-05-06-multi-step-bootstrapping/ While eligability traces are usually associated with multi-step bootstrapping, there is a lot to be said for multi-step bootstrapping techniques on their own. These techniques unify temporal-difference learning and monte-carlo methods. Often performing better in the intermediate steps than TD or MC, the two extremes of the multi-step variants. Quoting Barto and Sutton, “another way of looking at multi-step methods is that they free you from the tyranny of the time step. https://blog.stites.io/posts/2017-02-15-prism-is-a-lens-of-either/ Wed, 15 Feb 2017 00:00:00 +0000 https://blog.stites.io/posts/2017-02-15-prism-is-a-lens-of-either/ My coworker quickly explained one of the more esoteric components (Prism) quite susinctly today: Prism is a Lens s a where a may or may not be in s. A good example of this is that Prism is like Lens (Either a b) a. I’m not going to say that this is a hard and fast rule to live by, but I hope it’ll provide a good starting point for some future Lens studies. https://blog.stites.io/posts/2017-01-02-minkowski-error/ Mon, 02 Jan 2017 00:00:00 +0000 https://blog.stites.io/posts/2017-01-02-minkowski-error/ Normally to handle outliers we use Turkey’s method to detect extreme values. Multivariate methods also exist to detect and remove outliers. Minkowski error, however, allows us to keep our outliers and simply reduce the impact they have on our data. Instead of measuring our model using the squared error, we raise error to a power less than two: say 1.5. In this way, the contribution that an outlier gives is lessened and we can keep the data. https://blog.stites.io/posts/2016-11-12-double-brackets/ Sat, 12 Nov 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-11-12-double-brackets/ writeStuff :: [Char] -> IO () writeStuff filename = bracket openFiles closeFiles doStuff where openFiles :: IO (Handle, Handle) openFiles = do f0 <- openFile (filename ++ "_first") WriteMode f1 <- openFile (filename ++ "_second") WriteMode return (f0, f1) closeFiles :: (Handle, Handle) -> IO () closeFiles (f0, f1) = do hClose f0 hClose f1 doStuff :: (Handle, Handle) -> IO () doStuff = undefined -- etc Bad! don’t duplicate your brackets! https://blog.stites.io/posts/2016-11-11-typefamily-gotchas/ Fri, 11 Nov 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-11-11-typefamily-gotchas/ Typeclasses are an amazing abstraction in haskell. They are one of the fundamental building blocks in sharing and managing code. Among other points of mention, some of the benefits typeclasses have over OO-styled interfaces include the ability to create objects out of nothing (like mempty for the Monoid typeclass) and the ability to lazily add polymorphism at compile time (ie: fromInteger :: Integer -> a in which we define some fromInteger 1 :: Double later on in the program. https://blog.stites.io/posts/2016-11-07-intrinsic-fear/ Mon, 07 Nov 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-11-07-intrinsic-fear/ Cleverly posted before the US election. This paper talks about the introduction of “Intrinsic Fear” for DRL algorithms. The idea here is that DRL is being used in more and more life-or-death scenarios, but function approximation will make a RL algorithm to forget an infrequent occurence and repeat the mistake — turning into the Sisyphean Curse. They basically introduce a supervised “danger model” alongside their DQN implementation which models the analogy to “catastrophic failure” — I’m thinking something to the effect of “black swans. https://blog.stites.io/posts/2016-10-31-record-wildcards/ Mon, 31 Oct 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-10-31-record-wildcards/ Primarily, these can be seen as an antipattern because the user will introduce them into scope and forget about them. This is unnessecary and makes for a very easy thing to program the first time, but not nessecarily something that will be easy to maintain. https://blog.stites.io/posts/2016-10-29-haskell-coerce/ Sat, 29 Oct 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-10-29-haskell-coerce/ haskell has a coerce function. It’s pretty cool! in the haskell wiki, it talks about how newtypes incur no runtime cost, but when you have an array of newtypes, a runtime cost will still happen in order to instantiate the array of newtypes. Basically, a linear operation of noops will occur. This can be avoided by using coerce. import Data.Monoid import Data.Coerce -- Without coerce, a linear cost: x, y, z :: Sum Int x = [Sum 1, Sum 7, Sum 10000] -- which is basically the same as: y = map Sum [1,7,10000 :: Int] -- With coerce, it becomes an O(1) operation: z = coerce [1,7,10000 :: Int] https://blog.stites.io/posts/2016-10-26-count-min-sketch/ Wed, 26 Oct 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-10-26-count-min-sketch/ A lesser known probabalistic data structure (alongside bloom filters and the hyperloglog) includes the Count-Min Sketch. To cite Cormode and Muthukrishnan in their most recent paper “Approximating Data with the Count-Min Data Structure” [1] to illustrate the reason behind sketches Consider a popular website which wants to keep track of statistics on the queries used to search the site. One could keep track of the full log of queries [but the log will] become very large. https://blog.stites.io/posts/2016-09-30-tracing-in-haskell/ Fri, 30 Sep 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-09-30-tracing-in-haskell/ Today I learned about traceM and traceShowM. First off: wow! debugging is so much cleaner now! Secondly, this is important because with trace you are at the will of lazy evaluation - so if you are running things monadically, you now have the ability to garuntee more ordering of your statements. Talk about cool. https://blog.stites.io/posts/2016-06-19-regex-til-repeated-captures/ Sun, 19 Jun 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-06-19-regex-til-repeated-captures/ I consider myself decent with regexes; this comes from my data science days, when I was doing a lot of data munging at TurboSquid. I’ve always considered repeatition in capture groups “hard-mode” and usually skipped over them, as they didn’t have too much utility in my day-to-day. Today someone asked me to write one, and here’s the breakdown of how to think about it. Regular-expression.info has a page on “Repeating a Capturing Group” vs. https://blog.stites.io/posts/2016-05-18-consensus-algorithms/ Wed, 18 May 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-05-18-consensus-algorithms/ You can most easily think of a consensus algorithms arising from the need to replicate state machines. From this perspective, a state machine would live on a server, and identical copies of the same state machine would be copied to all other servers. In this way, even if a server goes down, the state machine can be accessed. These state machines are typically represented in logs Logs are the history of actions which the state machine uses to execute commands in the given order. https://blog.stites.io/posts/2016-04-12-data-text/ Tue, 12 Apr 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-04-12-data-text/ Surprise! UTF-16 is faster to parse than UTF-8. This is because UTF-8 requires either 8, 16, 24 or 32 bits (one to four octets) while UTF-16 requires either 16 or 32 bits to encode a character. As an aside, UTF-32 always requires 32 bits. As far as processing time is concerned, text with variable-length encoding such as UTF-8 or UTF-16 is harder to process if there is a need to find the individual code units, however between the two UTF-16 is less variable and thus has more predictable memory access — making it the faster of the two. https://blog.stites.io/posts/2016-04-11-knns/ Mon, 11 Apr 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-04-11-knns/ Just a little refresher on KNNs as I swing back into thinking about statistics and data science. KNNs are fairly compute-intensive since they do a full search through your entire training set to find a single datapoint. Usually the mean for regression and the mode for classification. Usually one would use euclidean distance in your calculations, however you must always be aware of whether or not your attributes are all normalized and of the same scale. https://blog.stites.io/posts/2016-04-10-memory-mapped-files/ Sun, 10 Apr 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-04-10-memory-mapped-files/ Memory-mapped files are a segment of virtual memory that we assign with direct correlation to some file (or “resource”). Applications can then treat the mapped portion of the file as if it were primary memory! It seems like the main upsell is that the operating system has a good api for this and, in java, we pass control to the operating system to take care of reading and writing the files. https://blog.stites.io/posts/2016-02-18-found-the-y-combinator/ Thu, 18 Feb 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-02-18-found-the-y-combinator/ The y-combinator is fix. As seen in the sockets tutorial I rewrote. The funny thing is that we use the y-combinator to loop a function indefinitely and, as it turns out, forever is the encouraged way to do this. fix is a little old-school. https://blog.stites.io/posts/2016-02-18-free-your-frees/ Thu, 18 Feb 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-02-18-free-your-frees/ I was working on a project of Alex Crough’s at the SF Haskell Hack Night which takes the Free monad to a whole new level. The primary reasoning behind Elision was that Free is actually not “free” enough when it comes to constraints and testing. An Elision doesn’t require that you pass it a functor, and on top of this, it’s also possible to “hot-swap” (or “mock” if you prefer) implementation details while unit-testing. https://blog.stites.io/posts/2016-01-29-left-for-errors/ Fri, 29 Jan 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-01-29-left-for-errors/ A code snippet from earlier – it’s a little funny since I was talking to dan chao who does a lot of scalaz, but no haskell, and we couldn’t pinpoint exactly why Left is used to pass errors. If we all wrote right-to-left I guess we’d be doing things the other way around. module Sum where data Sum a b = First a | Second b deriving (Show, Eq) -- OH! https://blog.stites.io/posts/2016-01-24-infix-mappend/ Sun, 24 Jan 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-01-24-infix-mappend/ Don’t forget the infix mappend: <>. As bytemyapp opinionates: remembering this kind of notation is important for genericifying your code. https://blog.stites.io/posts/2016-01-20-orphan-instances/ Wed, 20 Jan 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-01-20-orphan-instances/ Typeclasses have unique pairings of the class and the instance for a particular type. An orphan instance is a big deal. It’s when you have an instance for a datatype and a typeclass, but the instance is not in the same module as said datatype or typeclass. Why is this important? Well if you don’t own the typeclass/datatype you can newtype it! Even when you think you want an orphan instance so that you can have multiple instances for the same type, you still actually want to use newtype. https://blog.stites.io/posts/2016-01-19-vim-to-remember/ Tue, 19 Jan 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-01-19-vim-to-remember/ It’s easy to get stuck doing only what you already know in vim, so here are some from vimgolf – which I highly recommend – that I’ve found pretty cool: . repeat the insert-mode command from normal-mode <C-@> is a null character, apparently - but it also seems to repeat the last added character. Like . when in insert-mode. ctags will let you use <C-]> to jump to definitions across a project! https://blog.stites.io/posts/2016-01-12-polymorphic-types/ Tue, 12 Jan 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-01-12-polymorphic-types/ 12 Jan 2016 - Belmont, CA In looking up some information on Hindley-Milner, I came across this insightful point: An expression which is in multiple sets could have zero, one, or many types. For an expression to have many types is what it means to be polymorphic. (Source) Which may seem blatantly obvious, but when put in the context of: https://blog.stites.io/posts/2016-01-06-whats-imported-from-cabal-repl/ Wed, 06 Jan 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-01-06-whats-imported-from-cabal-repl/ 06 Jan 2016 - Belmont, CA If you have only an executable declared in your cabal file, then when you cabal repl it will import all(?) custom modules depended on in your main file, as well as a Main module which contains your main file. If you have a library exposed, then your cabal repl will only import your exposed modules. You also won’t get access to the magically generated Main module after you drop into ghci. https://blog.stites.io/posts/2016-01-05-browse-ghci/ Tue, 05 Jan 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-01-05-browse-ghci/ 05 Jan 2016 - Redwood City, CA Use :browse <module> to view modules which you haven’t yet imported. For example: Prelude> :browse Data.Bool bool :: a -> a -> Bool -> a (&&) :: Bool -> Bool -> Bool data Bool = False | True not :: Bool -> Bool otherwise :: Bool (||) :: Bool -> Bool -> Bool A perfect complement to my current workflow which includes the hoogle-GHCi integration. https://blog.stites.io/posts/2016-01-05-jwts-nobrainer/ Tue, 05 Jan 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-01-05-jwts-nobrainer/ Why? Well they’re self signed, so no storage is involved. Even if you are integrating systems, you don’t have to worry (too much) about “merging jwt-based authentication systems” – only then data that is contained on the tokens; but that was always going to be an issue. Even taking into account the fact that user information may be stored in your tokens isn’t a problem since you’d need the server’s private key to decrypt them. https://blog.stites.io/posts/2016-01-04-401-invokes-popup/ Mon, 04 Jan 2016 00:00:00 +0000 https://blog.stites.io/posts/2016-01-04-401-invokes-popup/ “401 Unauthorized” invokes a browser popup as it’s the default behaviour on almost every browser (that I know of). You can either return a non-401 status code, or change the Authorization header from Basic or Digest to something like Json. https://blog.stites.io/posts/2015-12-29-overloadedstrings/ Tue, 29 Dec 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-12-29-overloadedstrings/ 29 Dec 2015 - In the air, above Nevada When the OverloadedStrings pragma is enabled, it changes the definition of String from: type String = [Char] to: IsString a -> a with IsString defined as: class IsString a where fromString :: String -> a This means that strings are now higher-kinded types and, in order to use them, we need to make them concrete by either (a) explicitly typing what we intend them to be, or (b) using the ExtendedDefaultRules languages extension. https://blog.stites.io/posts/2015-12-20-the-magic-phrase-for-beginners-mindset/ Sun, 20 Dec 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-12-20-the-magic-phrase-for-beginners-mindset/ “If i didn’t know any better” - the phrase you can say at any point that gives you permission to say something silly with a beginner’s mind. https://blog.stites.io/posts/2015-12-18-git-autocompletion/ Fri, 18 Dec 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-12-18-git-autocompletion/ Just found a nifty little way to set up autocompletion in git. Check out this Code Worrier post. As a result, I’ve also come across just a few bash plugin managers and gave them all whirl. They’re all awefully slick but I think when it comes to bash, much like in the C/C++ communities, you wind up with a top-heavy apple-core population; the community is either very new and build out package managers like these, or they are very weathered/experienced and no longer feel the need for this kind of tool. https://blog.stites.io/posts/2015-12-17-higher-kinded-types/ Thu, 17 Dec 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-12-17-higher-kinded-types/ “higher-kinded types” exist because we have type constructors in haskell. So, putting my jvm hat back on, imagine interface constructors which take interfaces and types as arguments. This requires us to expand our language: type constuctors: *->* type constants (types with no constructor): * kind * is the kind of all standard lifted types. Primatives have type # and are unlifted. lifted types are repersented by a pointer (think C) and include most datatypes you’ll ever use. https://blog.stites.io/posts/2015-12-16-lifting-data-constructors/ Wed, 16 Dec 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-12-16-lifting-data-constructors/ When constructing a new data type, it makes sense that we’d want to have some validation to ensure that our inputs are correct. Say we have some type Name = String type Age = Integer data Person = Person Name Age deriving Show Clearly, we want to make sure that someone’s age isn’t less than 0 – and I’m pretty sure we also don’t want to have a Bobby Tables situation on our hands. https://blog.stites.io/posts/2015-12-15-eq-your-errors/ Tue, 15 Dec 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-12-15-eq-your-errors/ When building out error types in Haskell, we must derive Eq or the compiler can’t pattern-match on them. One instance of this is for using case expressions where the constructor is the condition. Thus, if the error’s constructor is the condition, we must have a way to compare constructors. We could, technically, build our own custom Eq definition, but for something as simple as errors, why would you want to? https://blog.stites.io/posts/2015-12-07-contra-co-variants/ Mon, 07 Dec 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-12-07-contra-co-variants/ A small little note to remember about contra- and co- variants for the future, as well as a mnemonic to remember which is which. The jist of it is that a contravariant can be thought of a wildcard of a class and all of it’s inherited types, while a covariant is a wildcard of a class and all of its subtypes. This leads to a couple of conclusions; the first being that, in a language like java, the contravariant has a terminal type - Object - while the covariant has a non-terminal type. https://blog.stites.io/posts/2015-12-02-kolmogorov-smirnov/ Wed, 02 Dec 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-12-02-kolmogorov-smirnov/ 02 December 2015 - San Francisco A cumulative distribution function (CDF) is the theoretical function which will tell you that some random variable can be found with a probability less than or equal to some value, p - which is normally found on the y-axis. The empirical cumulative distribution function (or EDF), on the other hand, is just your observed CDF. Where CDFs increment in steps of 1/n for an n-sized sample, an EDFs is usually much more sporadic, taking on the look of a step function with the horizontal sections representing where your data is most sparse. https://blog.stites.io/posts/2015-11-24-deep-work/ Tue, 24 Nov 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-11-24-deep-work/ I’m a big fan of Cal Newport’s work and, recently, have laid out a three-month plan to immerse myself more deeply in becoming a better engineer through functional programming, and to experimenting in the pursuit of machine learning research as my specialized expertise. This really isn’t anything new. This three-month plan was a comprehensive breakdown and milestone planning of CSSR, which I introduced in a previous post. Primarily, I was able to scope out some pragmatic deadlines and reach some good conclusions: including that publication comes before everything else and that I will write CSSR in scala first to offset the added cost of learning the extremely pure functional programming paradigm, then keep an experimental feature branch running of the same work in Haskell - even though it might blow the scope of this project out of proportion. https://blog.stites.io/posts/2015-10-16-defender-methods-and-multiple-inheritance/ Fri, 16 Oct 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-10-16-defender-methods-and-multiple-inheritance/ 09 Oct 2013 - San Francisco, CA We’ve started the shift to Java 8, over at Bina, and I have been spearheading the a lot of the fresh code. As someone who thinks and reads up on Haskell almost daily at this point. I’d like to wrap up some of my older, backlogged blog posts regarding the jvm and start shifting focus to things I’m more interested in. As far as I’ve gone with java 8, a number of features have proven invaluable in keeping our code clean and flexible include some of the new interface functionality: defender methods, static methods, and functional interfaces. https://blog.stites.io/posts/2015-10-09-why-causal-states/ Fri, 09 Oct 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-10-09-why-causal-states/ 10 Oct 2015 - San Francisco, CA I’ve started work on cleaning up some loose ends from Cosma and Kristi’s CSSR (Causal State Splitting Reconstruction) paper, “Blind Construction of Optimal Nonlinear Recursive Predictors for Discrete Sequences” - arxiv:cs.LG/0406011. In this post I want to prime you with the environment of the algorithm at a high-level, as well as go over some possible motivating examples of what makes CSSR interesting. Let’s say that we are examining some finite stream of data that is seemingly random, and want to find the hidden state which determines how this stream is being output. https://blog.stites.io/posts/2015-10-06-haskell-nary-trees/ Tue, 06 Oct 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-10-06-haskell-nary-trees/ 06 Oct 2015 - San Francisco In the start of my work on CSSR, I have had to construct n-ary trees. Surprisingly, this took a lot or realization to get to, since most examples in haskell are showered with binary trees, so it’s easier to forget their n-ary variants. Furthermore, most n-ary implemintations that I’ve come across don’t seem to give nodes any value, except at the lowest level. Well, that last part may seem a little insane, but perhaps this was because a majority of my searching was on stackoverflow. https://blog.stites.io/posts/2015-10-06-on-genetic-algorithms/ Tue, 06 Oct 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-10-06-on-genetic-algorithms/ 06 Oct 2015 - San Francisco From Talking Machines, summarized: “I wanted to elaborate on how Genetic algorithms may seem great, but in a fight between pure randomness and intelligent design - going the path of intelligent design is always best.” https://blog.stites.io/posts/2015-09-27-eclipse-hanging-git-status/ Sun, 27 Sep 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-09-27-eclipse-hanging-git-status/ Sept 27, 2015 - Redwood City, CA If eclipse starts hanging with the process “Computing git status for repository…” remove the following files: rm -r $ECLIPSE_WORKSPACE_DIR/.metadata/.plugins/org.eclipse.core.resources/.projects/*/org.eclipse.egit.core rm -r $ECLIPSE_WORKSPACE_DIR/.metadata/.plugins/org.eclipse.core.resources/.projects/*/.indexes/properties.index https://blog.stites.io/posts/2015-05-03-trait-internals/ Sun, 03 May 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-05-03-trait-internals/ In the end, Scala needs to translate traits into classes and interfaces for the JVM to be able to handle them. A trait with only abstract methods is converted into a Java Interface. If a trait has concrete fields, a companion class is created alongside the interface whose static fields hold the trait’s values. trait ConsoleLogger extends Logger { val maxLength = 15 def log(msg: String) { println(msg) } } turns into https://blog.stites.io/posts/2015-04-25-self-types-in-traits/ Sat, 25 Apr 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-04-25-self-types-in-traits/ 25 April 2015 - Mountain View Last time we learned that a scala trait might extend a class. Well — that might offer up a little confusion when the superclass for any of those mixins references when looking at self references. For this reason, scala has self types. When a trait starts with this:_Type_ => then it can only be mixed into a subclass of the given _Type_. This is similar to a trait with a supertype, when invocations to super were post-fixed with the supertype in brackets (super[_SuperType_]. https://blog.stites.io/posts/2015-04-19-trait-similarity-to-classes/ Sun, 19 Apr 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-04-19-trait-similarity-to-classes/ Traits can have constructors! Wow, I wasn’t expecting that one. Just like a class, the constructor is embedded in the body of the trait itself. Every trait has one parameterless constructor. The fact that a trait has a parameterless constructor is the only technical difference between traits and classes. So anything you can do with a class, you can do with a trait! trait Foo { def bar(x: String) def log(y: String) {} // part of the constructor val baz = 10 out. https://blog.stites.io/posts/2015-04-07-trait-fields-in-scala/ Tue, 07 Apr 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-04-07-trait-fields-in-scala/ 07 April 2015 - Reykjavik, Iceland I mentioned in an earlier post that methods declared, but not defined, are implicitly defined as abstract: trait Foo { def bar(x: String) // abstract definition def baz(y: String) {} // concrete definition } The same goes with fields: trait Foo { val bar:Int // abstract field val baz = 10 // concrete field } When a class inherits from a trait, these fields are placed into the subclass - not inherited. https://blog.stites.io/posts/2015-04-02-abstract-methods-in-scala-traits/ Thu, 02 Apr 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-04-02-abstract-methods-in-scala-traits/ 02 April 2015 - Reykjavik, Iceland When you want to override an abstract method in a traits, you must declare both abstract on top of our override in order for this to work: trait Logger { def log(msg: String) // abstract } trait WarnLogger { abstract override def log(msg: String){ super.log(s"[WARN] $msg") } } If you fail to add the abstract modifier, Scala will think that the log method is concrete, will find an abstract log instead, and the compiler will error. https://blog.stites.io/posts/2015-03-28-trait-inheritance/ Sat, 28 Mar 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-03-28-trait-inheritance/ 28 March 2015 - Reykjavik, Iceland You can add multiple traits that invoke eachother, starting with the last one. trait WarnLogger extends Logger { override def log(msg: String) { super.log(s"[WARN] $msg") } } trait ShortLogger extends Logger { val maxLength = 10 override def log(msg: String) { super.log( if (msg.length <= maxLength) msg else msg.substring(0, maxLength - 3) + "..." ) } } Both modify message being passed to Logger. https://blog.stites.io/posts/2015-03-13-overriding-traits-during-instantiation/ Fri, 13 Mar 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-03-13-overriding-traits-during-instantiation/ 13 March 2015 - Redwood City You can create a base trait with concrete methods which will be used in a class only to override that class with a different trait on impliementation. For example: trait Logger { def log(message: String) { } // a concrete implimentation } Used in a class definition: class CustomerAnalytics extends Analytics with Logger { def foo(input: Double) { if (input > max) log("Insufficient funds") // . https://blog.stites.io/posts/2015-03-07-introduction-to-scala-traits/ Sat, 07 Mar 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-03-07-introduction-to-scala-traits/ 07 March 2015 - Mountain View I’ve been digging into scala more and more. Inspired by Horstmann’s Scala for the Impatient, I’m writing a series of posts on parts of Scala I find interesting. Some of the examples I show will come directly from Horstmann, but only because they’re pretty great examples. I highly recommend the book. Scala traits are pretty cool. They’re basically the same as interfaces in Java. Except for the fact that they’re a lot more flexible than interfaces and they’ve been made to address the multiple inheritence problem which is missing in Java. https://blog.stites.io/posts/2015-01-06-sparknotes-from-rob-eisenbergs-article/ Tue, 06 Jan 2015 00:00:00 +0000 https://blog.stites.io/posts/2015-01-06-sparknotes-from-rob-eisenbergs-article/ 06 January 2015 - Redwood City This is a pretty long article written by a former member of the Angular2.0 team. The counter at the bottom estimates an hour-long read, so I figure’d I’d write some (as objective as possible) sparknotes: Motivations for Angular 2.0 When AngularJS was first created, almost five years ago, it was not originally intended for developers. The web is no longer a jQuery cesspool 5 years ago (angular’s creation) mobile was barely a thing Learning angular looks like this Features and Design of Angular 2. https://blog.stites.io/posts/2014-12-31-languages-for-2015/ Wed, 31 Dec 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-12-31-languages-for-2015/ 31 December 2014 - Mountain View, CA Inspired by Richard Astbury’s post, entitled Langages, I’ve come up with my own take on the state of languages for 2015 with respect to what I’d like to work on this year. As with Astbury, I’ll add a disclaimer that this post is “opinionated, poorly researched, and really just the thoughts in my head.” Going to start with the same list. Cheers! C# No intention of going in this direction. https://blog.stites.io/posts/2014-12-19-john-carmak/ Fri, 19 Dec 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-12-19-john-carmak/ 19 December 2014 - SFO Just shook hands with John Carmak on my way to Dallas - a connecting flight. It felt like some kind of cameo in a great film, with all the things happening recently, for instant Bina getting acquired by Roche. It was a fantastic moment. https://blog.stites.io/posts/2014-11-11-bash-statistics-scripts/ Tue, 11 Nov 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-11-11-bash-statistics-scripts/ 11 Nov 2014 - Redwood City ================= $ sort -n allPass.qual | uniq -c > tmp $ sed -n '$=' tmp 17917 cut -f 3 test.sorted.gatkDP.sample_interval_summary tab is default delimninator #too long: $ sed -i.bak -E -e 's/^([0-9]+)$/\1\.0/g;s/([0-9]+\.[0-9])[0-9]+/\1/g' allPass.qual $ sort -n allPass.qual | uniq -c > tmp $ sed -i.bak -E 's/([0-9]+) (-?[0-9]+\.?[0-9]*)/\[\2\,\1\]\,/g' tmp $ sed -n '$=' tmp 5335 $ cat tmp > allPass.qual $ echo ']' >> allPass. https://blog.stites.io/posts/2014-11-10-bash-statistics/ Mon, 10 Nov 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-11-10-bash-statistics/ 10 Nov 2014 - Redwood City I had a case where I needed to generate some simple statistics from an indel file which one of our bioinformaticians had procured for me. These files are huge, but instead of asking for the parsed file, I thought it might be fun to do this in bash. I’ll walk through an example. Lets say I have a text file, filename, with the following content: https://blog.stites.io/posts/2014-09-24-clean-out-git-branches/ Wed, 24 Sep 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-09-24-clean-out-git-branches/ 24 September 2014 - Redwood City, CA Thought I’d share this little tidbit — I now routinely clean out my git branches with: git branch | xargs -n 1 git branch -d Of course, this only removes branches you haven’t merged, so it’s completely safe. I set up a cronjob to run to run this in my primary repo about once a month! https://blog.stites.io/posts/2014-09-04-bootstrap3-downgrader-in-sed/ Thu, 04 Sep 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-09-04-bootstrap3-downgrader-in-sed/ 04 Sept 2014 - Redwood City, CA Some notes on sed: always use -E for the more ‘standard’ regex always use single quotes for the regex so that you don’t have to escape metacharacters sed doesnt allow for non capture groups you can execute a script with -e (not nessecary for a regex) chop up your regexs into smaller ones and use -e to seperate out regexs by ;s you can use -i to change a file on the fly. https://blog.stites.io/posts/2014-08-11-idempotence-in-programming/ Mon, 11 Aug 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-08-11-idempotence-in-programming/ 08 August 2014 - Redwood City, CA In mathematics, the concept of idempotence implies that an operation can be executed any number of times and still yield the same result as if it was only run once. f(f(x)) = f(x). Programming-wise, the word means that some method or function which mutates a variable has no additional effect on the variable if it is called on it more than once, given the same input parameters. https://blog.stites.io/posts/2014-08-09-simple-ng-scope-inheritance/ Sat, 09 Aug 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-08-09-simple-ng-scope-inheritance/ 09 August 2014 - San Francisco, CA This is the second post I’ve written from rebuilding angular under the guidance of Tero Parviainen’s Build your own Angularjs. Last time, I went through a quick-and-dirty explaination of the digest loop and today it’s scope inheritance. If you don’t settle for perculated wisdom, like myself, I’d recommend buying the book. However if all you want is the high level to get that understanding in, then hopefully these posts should be good enough. https://blog.stites.io/posts/2014-08-01-case-sensitive-git/ Fri, 01 Aug 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-08-01-case-sensitive-git/ 01 August 2014 - San Francisco, CA By default - at least on macs - git is case insensitive, but case preserving. Meaning that the first time you write a file is how that file will live for the duration of its life in the git tree. It’s supposed to help you, but I’ve never used any of this to my benefit. Arguably, case-sensitivity and that kind of granular attention-to-detail should be in the nature of software development. https://blog.stites.io/posts/2014-07-22-simple-ng-the-digest-loop/ Tue, 22 Jul 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-07-22-simple-ng-the-digest-loop/ 22 July 2014 - San Francisco, CA Of late, I’ve been rebuilding angular by following Tero Parviainen’s Build your own Angularjs. After finishing up work on the digest loop, I figure it would be best to go through what I’ve learned so far. What is the digest loop? It’s really beautifully simple. The digest loop is simply an eventing system that maintains angular’s scopes for databinding. This is, in my opinion, one of the building blocks of what it means to build something in ’the angular way. https://blog.stites.io/posts/2014-07-01-git-reset-a-single-file/ Tue, 01 Jul 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-07-01-git-reset-a-single-file/ 01 July 2014 - Redwood City, CA I feel like, at a certain point in time, I used to be able to run the command git reset --hard path/to/file. However this might have been me only working with single files, because now I frequently get the error: fatal: Cannot do hard reset with paths. This one has been a tiny thorn in my side - perhaps ever since I upgraded to git 2. https://blog.stites.io/posts/2014-06-17-backbone-to-angular-part-ii/ Tue, 17 Jun 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-06-17-backbone-to-angular-part-ii/ 17 June 2014 - Redwood City, CA Lets talk about tests. Yes, tests. Some of you might love the phrase, and some of you may hate it. When it comes to refactoring, however, if you’re not using tests you’re pretty much going to screw yourself somewhere along the road. Even if you’re not a huge fan of TDD, doing test-driven refactors are going to save you a lot of hair pulling down the road. https://blog.stites.io/posts/2014-06-12-ammend-your-commits/ Thu, 12 Jun 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-06-12-ammend-your-commits/ 12 June 2014 - Redwood City, CA To alter the last commit Remember that git commit --amend will add forgotten changes to the last commit in your logs. Don’t do this if you’ve already pushed the commit somwhere, unless you know what you’re doing. To alter a commit further back Other notes that I’ve collected over the internet. If you are looking to change a commit further back in history, look into git rebase. https://blog.stites.io/posts/2014-06-11-backbone-to-angular-part-i/ Wed, 11 Jun 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-06-11-backbone-to-angular-part-i/ 11 June 2014 - Redwood City, CA At bina, we have two products: bina portal and bina annotation platform. While the annotation platform has been written from the ground-up in angular, the bina portal was written in backbone. It’s also one of those applications which has grown into a monolithic zombie over the years. It’s got to go, but it can’t be canned. Taking on the codebase headfirst isn’t really an option for us as it means that we will have a brand-new product which won’t be user-tested. https://blog.stites.io/posts/2014-05-17-a-useful-double-equals-in-javascript/ Sat, 17 May 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-05-17-a-useful-double-equals-in-javascript/ 17 May 2014 - San Francisco, CA Today I was reviewing John Resig’s Secrets of a Javascript Ninja Quiz and noticed in section 21 that, when checking values in a cache, he uses a double-equals sign to see if a value exists on the object. Check it out: function isPrime( num ) { if ( isPrime.cache[ num ] != null ) return isPrime.cache[ num ]; var prime = num != 1; // Everything but 1 can be prime for ( var i = 2; i < num; i++ ) { if ( num % i == 0 ) { prime = false; break; } } isPrime. https://blog.stites.io/posts/2014-04-16-node-streams/ Wed, 16 Apr 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-04-16-node-streams/ 16 April 2014 - San Francisco, CA Node has brought to us the ability to run javascript anywhere with the pretense that asyncronisity is getting our servers comparably as fast as a C++ or Go server. One of the key, unsung features which allows for such amazing speed is concepts of streams. Buffering will loads files/documents/objecs into memory, however streams can bypass this loading process and allow content to be directly funnelled into a server response, which happens to be a stream object! https://blog.stites.io/posts/2014-03-02-so-you-want-to-learn-web-development/ Sun, 02 Mar 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-03-02-so-you-want-to-learn-web-development/ 02 March 2014 - San Francisco A few of my friends have been asking me how to get started with learning how to program. Most of them come from Comp-sci backgrounds from undergrad, and want to start free lancing but have no idea how to start applying their skills in a formal real-world job setting. When you know a ton of smart people who want to split their time between their love for computer science and circus, you get a lot of interest in the digital nomad lifestyle. https://blog.stites.io/posts/2014-02-21-everyday-uses-of-javascript-design-patterns/ Fri, 21 Feb 2014 00:00:00 +0000 https://blog.stites.io/posts/2014-02-21-everyday-uses-of-javascript-design-patterns/ I’ve been reading Essential Javascript Design Patterns by Addy Osmani, when I realized that most design patterns are already common in everyday use of code. Inspired by this thought, I reread through my notes and decided to write out where we would see or hear about them on a daily basis. Module Pattern +IIFies, as well as RequireJS are also examples of the [module pattern][module pattern]. A variation of the module patterns, the [Module mixin][Module mixin] is used as the interface to such libraries as jQuery, Backbone, and… well basically every library out there - so long as they introduce nothing into the global scope. https://blog.stites.io/posts/2013-12-26-5-reasons-you-should-use-bookshelfjs/ Thu, 26 Dec 2013 00:00:00 +0000 https://blog.stites.io/posts/2013-12-26-5-reasons-you-should-use-bookshelfjs/ Back home for the holidays! Of late, I’ve gotten ahold of my first contract job which I am very excited about. There isn’t too much time to look into certain technologies that I’m really stoked about - distributed systems, more work with graphs, or writing a Restricted Boltzman Machine in Javascript - however I am eager to hammer out confidence in client work. One thing that did have me particularly down was that, due to the nature of the app, we’d be working with SQL. https://blog.stites.io/posts/2013-12-08-a-succinct-look-at-pseudo-classical-selectors/ Sun, 08 Dec 2013 00:00:00 +0000 https://blog.stites.io/posts/2013-12-08-a-succinct-look-at-pseudo-classical-selectors/ Tomorrow I’ll be presenting to my peers about pseudo-classical selectors in CSS. I find this pretty amusing since I tend to avoid making my websites look pretty - however since starting to attend Daniel Chao’s CSS classes in the after-hours of our sprints at HackReactor - I’ve started to find something pleasant about pretty websites. The takeaway so far has been: CSS is hard, voodoo, and mangled together in many instances - but if you put in just a few hours of attention into your CSS the benefits of a beautiful website can be amazing! https://blog.stites.io/posts/2013-11-26-bloom-filters-for-data-detectives/ Tue, 26 Nov 2013 00:00:00 +0000 https://blog.stites.io/posts/2013-11-26-bloom-filters-for-data-detectives/ Track I’m coding to: Soul Satisfaction by Bit Funk on hypem I just finished writing a Bloom filter over Thanksgiving break. What an incredible experience! Aside from learning how to whip up a bloom filter, this was also my first experience in using bitwise operators. I mean I’ve used bitwise operators before, but not for anything useful just little things like using ~~(n) instead of Math.floor(n) for positive numbers, and x << y as a replacement for x * Math. https://blog.stites.io/posts/2013-11-03-bnf-grammar/ Sun, 03 Nov 2013 00:00:00 +0000 https://blog.stites.io/posts/2013-11-03-bnf-grammar/ The latest thing I’ve done was construct a recursive-decent parser for JSON. The structure of constructing one of these uses a form called Backus-Naur Form, developed by Backus and Naur, however it is practically identical to Chomsky’s Universal Theory of Grammar. Both have the same format, were developed at the same time, and were developed independently! Great minds think alike! It’s pretty cool to look into, in short the idea is to break down larger structures - sentences - into ’tokens’ - nouns, verbs, and objects - and recursively go over the rules you define to generate something meaningful. https://blog.stites.io/posts/2013-10-17-backbone-twitter-clone-part-1/ Thu, 17 Oct 2013 00:00:00 +0000 https://blog.stites.io/posts/2013-10-17-backbone-twitter-clone-part-1/ I’ve just finished working through JavascriptIsSexy’s Learn Backbone.js Completely which, in my opinion, is one of the best places to learn javascript on the interwebs. I’ve also been reading John Resig’s Secrets of the Javascript Ninja, and have completed half of Learn Javascript Properly. So I may not be a javascript pro, still I’m gunning to build my first Backbone App. This is going to be a twitter clone, because it seems fairly comprehensive.