Tagged: programming
Prioritized Experience Replay – Open Question
I have an open question about Prioritized Experience Replay from [Schaul15]. From my experiments, it seems that an equation in the publication is wrong, but maybe I’m overlooking something. I’d appreciate input.
Gym Experiments: CartPole with DQN
To introduce ourselves to reinforcement learning with Deep-Q Networks (DQN), we’ll visit a standard OpenAI Gym problem, CartPole. We’ll cover deeper RL theory in a later post, but let’s get our hands dirty first, to build some intuition!
The complete series can be found on the bottom of this post and the latest version of the GitHub repo can be found here. Be sure to get set up before you begin.
Gym Experiments: Setting Up
Kicking off a series of posts on OpenAI’s gym environment, I’ll cover some light bootstrapping to get us up and running more quickly. I promise, it will be short and sweet! I’ll be referring back to this from later posts in the series.
The complete series can be found on the bottom of this post and the latest version of the GitHub repo can be found here.
Syntactic Archery
Something that trips me up occasionally in C# is syntactic sugar with properties. Specifically, the difference between the fat arrow (=>) and assignment (=) with properties. Here’s a quick note to help clear up any confusion.
Elm Animation Showcase
Here’s a quick demo of a showcase set of Elm animations. Check out the GitHub repo under the Showcase directory; run the app from the top-level Showcase.elm module.
I’ve been meaning to publish a full-featured post for a while, but I’ve gotten bogged down in my studies once again. In the interest of not letting this content get too stale, I’m simply publishing what I have.
This expands upon the prior work I’ve blogged about, with some modifications. At the time of this writing, it demonstrates how to wire up two separate components, each with a collection of different animations (some of which are keyed off of one another, or off of other actions). The two components transition back and forth, too.
- The first component is an animated username/password form.
- The second component is an animated list of items.
At this point, pretty much all work starts from the MinimalModules.elm file included in the repo and builds up from there. Just follow the comments and you should be able to hit the ground running, without worrying about the underlying piping.
I’ll likely expand this showcase set as I find time, but I won’t be updating this post.
Catching Completed Elm Animations
This post continues a series I’ve been writing as I toy with the mdgriffith/elm-style-animation Elm package. I’ve primarily been focused on both learning how to use the animation package, but as I’ve explored, there seems to be room for leveraging a higher layer of abstraction to more rapidly compose and deploy animations. Maybe things will work out, or maybe I’ll crash and burn. Either way, it has been a fun journey so far and I’ve enjoyed talking with the Elm community and the animation package’s author, Michael!
Let’s recap what we’ve covered so far. I’ll be mainly discussing changes that I’ve made to the previous code, so you’ll want to take a look at these posts (especially the first post) before we begin.
- In the first post, I discussed a basic framework for Elm animations that seemed to have fallen out naturally as I was manipulating data types for composing animations.
- In the second post, I extended this framework to perform more complex behavior like swapping view elements for controlling transitions in flow control.
In this post, I’m going to explore detecting the lifecycle of animations and triggering commands based on that lifecycle.
Read on for details, or just grab the code from this Github repo and run with it. You’ll want to look for the FinishAnimation.elm module.
Swapping Elm Animations
I recently wrote about a handy extension to layer animations on top of the standard Elm Architecture.
This post explores taking the basic animation framework and building more complex animations upon it.
Read on for details, or just grab the code from this Github repo and run with it. You’ll want to look for the SwapView.elm module.
Basic Framework for Elm HTML Animations
I’ve been dabbling a bit with the mdgriffith/elm-style-animation package for Elm 0.18 and so far it looks like quite a capable library. I did encounter some headwinds with layering multiple effects on the same HTML element at different times, so I scratched out a basic framework for extending animations. I’ll walk you through the basic extensions I’ve layered on top of the standard Elm Architecture.
What’s the goal of all this? Being able to define easily reusable animations, just like CSS classes! And being able to manage the complexity that comes along with creating many different animations.
Read on for details, or just grab the code from this Github repo and run with it.
Elm Development Environment
First off, my apologies for the deep silence of late. I’m pretty deep in the trenches when my Master’s semesters are running and I simply had to take a personal break during my summer. I’m hoping this post can help me pick up my cadence again!
I recently dove back into Elm development, only to find that the community has evolved rapidly between versions 0.16 and 0.18 (current as of this writing). I thought I’d share my notes on how I have my new development environment set up, now that we have more options beyond simple text editors (or vim).
Right now, my basic environment looks like this:
- Running on OS X Sierra
- Elm 0.18
- Visual Studio Code
- Sascha Brink’s Elm Language Support extension for VS Code
That’s all I’m using to get a nice rapid development environment up and running.
Work-around for “commitAndReleaseBuffer” Compilation Error in Elm
I ran into a compilation bug when following the “Task Tutorial” in Elm’s Reactivity documentation.
This is the output I got when compiling (where Main.elm is the exact copy/paste from the “HTTP Tasks” section of the docs):
> elm-make Main.elm --output=index.html Success! Compiled 1 modules. elm-make: index.html: commitAndReleaseBuffer: invalid argument (invalid character)
From browsing around, it looks like the issue might be that the Elm compiler needs to set a LANG property in the Haskell code that runs the compiler to better specify encodings. Whether it’s relevant or not to the default encoding that failed here, I’m based in the US.
While the bug is still open, read on for a work-around, if you encounter the same issue.
