Stacks on Stacks

The Serverless Ecosystem Blog by Stackery.

Posts on Engineering

The Lazy Programmer’s Guide to Web Scrapers
Anna Spysz

Anna Spysz | March 21, 2019

The Lazy Programmer’s Guide to Web Scrapers

I am a proud lazy programmer. This means that if I can automate a task, I will absolutely do it. Especially if it means I can avoid doing the same thing more than once. Luckily, as an engineer, my laziness is an asset - because this week, it led me to write an HTML scraper for our Changelog, just so I wouldn’t have to manually update the Changelog feed on our new app homepage (btw, have you seen our new app homepage? It’s pretty sweet).

Automate the Boring Stuff book cover

If only I could also make robots mow my lawn...

Why automation?

I am a firm believer in lazy programming. That doesn’t mean programming from the couch in your PJs (though that’s what work from home days are for, amirite?). I mean working smarter rather than harder, and finding ways to automate repeated processes.

One of our more recent processes was adding a Changelog to our Docs page, to let our users know what’s going on with Stackery and so we can announce major updates. However, the initial Changelog update process was anything but lazy:

  • Create a Changelog doc
  • Yell at engineers for not updating it*

Yup, room for improvement.

A better way of doing things

After much thought and some trial and error, we came up with a better system. Here’s what our new Changelog process looks like, in a nutshell:

  • Each engineer updates an internal Changelog file in an individual repo as part of the PR process
  • When the PR is merged, a scraper Lambda function is triggered by a Git webhook and compiles the new Changelog item into a single doc
  • That doc is then reviewed a few times a week and the most interesting items are written up in our public Changelog (this is the unavoidably manual part of the process, until we build an AI that writes puns, anyway)
  • The public Changelog is scraped by our changelog scraper Lambda and its four most recent items are pulled, reformatted, and displayed on the app homepage

Here’s the scraper function I built that does the last part of the process:

const AWS = require('aws-sdk');
const cheerio = require('cheerio');
const rp = require('request-promise');
const changelogUrl = 'https://docs.stackery.io/en/changelog/';
const s3 = new AWS.S3();

exports.handler = async () => {
  // options for cheerio and request-promise
  const options = {
    uri: changelogUrl,
    transform: function (html) {
      return cheerio.load(html, {
        normalizeWhitespace: true,
        xmlMode: true
      });
    }
  };
  // remove emojis as they don't show up in the output HTML :(
  function removeEmojis (string) {
    const regex = /(?:[\u2700-\u27bf]|(?:\ud83c[\udde6-\uddff]){2}|[\ud800-\udbff][\udc00-\udfff]|[\u0023-\u0039]\ufe0f?\u20e3|\u3299|\u3297|\u303d|\u3030|\u24c2|\ud83c[\udd70-\udd71]|\ud83c[\udd7e-\udd7f]|\ud83c\udd8e|\ud83c[\udd91-\udd9a]|\ud83c[\udde6-\uddff]|[\ud83c[\ude01\uddff]|\ud83c[\ude01-\ude02]|\ud83c\ude1a|\ud83c\ude2f|[\ud83c[\ude32\ude02]|\ud83c\ude1a|\ud83c\ude2f|\ud83c[\ude32-\ude3a]|[\ud83c[\ude50\ude3a]|\ud83c[\ude50-\ude51]|\u203c|\u2049|[\u25aa-\u25ab]|\u25b6|\u25c0|[\u25fb-\u25fe]|\u00a9|\u00ae|\u2122|\u2139|\ud83c\udc04|[\u2600-\u26FF]|\u2b05|\u2b06|\u2b07|\u2b1b|\u2b1c|\u2b50|\u2b55|\u231a|\u231b|\u2328|\u23cf|[\u23e9-\u23f3]|[\u23f8-\u23fa]|\ud83c\udccf|\u2934|\u2935|[\u2190-\u21ff])/g;
    return string.replace(regex, '');
  }
  // parse the scraped HTML, then make arrays from the scraped data
  const makeArrays = ($) => {
    const titleArray = [];
    const linkArray = [];
  
    $('div > .item > h2').each(function(i, elem) {
      titleArray[i] = removeEmojis($(this).text());
    });
  
    $('div > .item > .anchor').each(function(i, elem) {
      linkArray[i] = $(this).attr('id');
    });
  
    return makeOutputHtml(titleArray, linkArray);
  }
  // format the arrays into the output HTML we need
  const makeOutputHtml = (titleArray, linkArray) => {
    let output = `<h2>Stackery Changelog</h2>
      <p>We're always cranking out improvements, which you can read about in the <a href='https://docs.stackery.io/en/changelog/'  target='_blank' rel='noopener noreferrer'>Stackery Changelog</a>.</p>
      <p>Check out some recent wins:</p>
      <ul>
      `;
    for (let [i, title] of titleArray.entries()) {
      // only get the latest four entries
      if (i < 4) {
        output += `<li><a href='https://docs.stackery.io/en/changelog/#${linkArray[i]}' target='_blank' rel='noopener noreferrer'>${title}</a></li>
        `;
      } 
    }
    output += `</ul>`;
    return output;
  };
  // request the changelog HTML
  const response = rp(options)
    .then( async ($) => {
      // create output HTML
      const outputHtml = makeArrays($);
      // set the bucket parameters
      const params = {
        ACL: 'public-read', // permissions of bucket
        Body: outputHtml, // data being written to bucket
        Bucket: process.env.BUCKET_NAME, // name of bucket in S3 - BUCKET_NAME comes from env vars
        Key: 'changelog/changelog.html' // path to the object you're looking for in S3 (new or existing)
      };
      // put the output HTML in a bucket
      try {
        const s3Response = await s3.putObject(params).promise();
        // if the file is uploaded successfully, log the response
        console.log('Great success!', s3Response);
      } catch (err) {
        // log the error if the file is not uploaded
        console.log(err);
        const message = `Error writing object ${params.Key} to bucket ${params.Bucket}. Make sure they exist and your bucket is in the same region as this function.`;
        console.log(message);
      } finally {
      // build an HTTP response
      const res = {
        statusCode: 200,
        headers: {
          'Content-Type': 'text/html'
        },
        body: outputHtml
      };
      return res;
      }
    })
    .catch((err) => {
      console.log(err);
    });

  return response;
};

Of course, any lazy serverless developer will use Stackery to avoid the tedium of writing their own YAML and configuring permissions, so naturally, I built this scraper in our app:

The stack in Stackery

It includes three cloud resources:

  • A Timer (AWS CloudWatch schedule event) that triggers the scraper function once a day
  • A Lambda function that pulls HTML content from our public Changelog, trims the data to just titles and links, outputs to a new HTML file and puts it in the designated S3 bucket
  • An S3 bucket that holds our HTML file, which is fetched by the app and displayed on the homepage

If you’d like to implement your own web scraper that outputs to an S3 bucket, grab the code from our GitHub repository - after all, the best lazy programmers start with others’ code!

* Just kidding. There’s no yelling at Stackery, only hugs.

A Greater Gatsby: Modern, Static-Site Generation
Toby Fee

Toby Fee | February 04, 2019

A Greater Gatsby: Modern, Static-Site Generation

Gatsby is currently generating a ton of buzz as the new hot thing for generating static sites. This has lead to a number of frequent questions like:

  • A static…what now?
  • How is GraphQL involved? Do I need to set up a GraphQL server?
  • What if I’m not a great React developer, really more of a bad React developer? Does this mean our copywriter won’t have to push to a GitHub repo to add a new page?

I had a few of these questions myself and decided to get firsthand experience by creating a few sites using Stackery and Gatsby. While I am good with Javascript and the general mechanics of websites, I am neither a React or a GraphQL expert. I’ve used both but as I enter my mid 30’s I find my mind is like a closet in Manhattan: it has room for only one season’s worth of interests.

What does Gatsby do exactly?

Gatsby is a static site generator, intended to build all the HTML you need based on data you created. Presumably, that data was simpler to manage than straight HTML.

I don’t use a static site generator now. Should I?

A static site generator probably doesn’t make sense if your site is truly static (e.g. a marketing site that gets updated twice a year.) Static site generators make more sense if you are building something that updates every 1-10 days, like a professional blog.

If you’ve ever used Jekyll, another static site generator, the basic concepts are similar: turn a folder of plain markdown files, or the like, into a complete site with just a bit of config.

But the limitations of Jekyll and other older generators are myriad:

  • Complex and endemic config formatting.
  • The lack of some particular feature which you need and they don’t have (it generates blog posts just great and interprets your ‘author’ field, but now you want posts two authors, for instance.)
  • There’s no great way to get new files from all team members. Often teams using Jekyll end up using a GitHub repository to store their source text files (again, Markdown or what have you) meaning your marketing copywriter needs to learn git to add a new blog post or more often: email a developer with an attached file for her to add.
  • You can’t make a Jekyll site in React.

Gatsby offers significant improvements in these areas since, along with bare-text files, Gatsby can import data from almost any data source and generate a clean React site based on the data it queries.

GraphQL is even cooler than React, but I have concerns.

GraphQL offers a tantalizing dream: a system that can connect multiple data sources with a simple query language. Suddenly your hip NoSQL databases and creaky SQL databases can all be displayed through a single app.

But building and running GraphQL are not always simple, and fundamentally this new engine requires learning a new query structure. It’s cool to see that Gatsby is ‘powered by GraphQL’ but doesn’t that mean there will be some massive friction in getting this deployed?

But remember Gatsby generates a static site. It uses GraphQL to access DB’s and generate your site, but once that process is over, GraphQL doesn’t need to be running for your site to work.

But isn’t there still a learning curve?

Yeah, that part is inescapable. While many common setups have templates to start with, your team’s weird content database from 1997 is going to require some custom config. There’s no way this could be harder than importing the data yourself, and once it’s configured you only need to update your database and re-run the Gatsby build.

What if I’m not a great React Developer?

If it took a React expert to use Gatsby, it wouldn’t really have a market and that’s clearly not the case. Ultimately, if we could make awesome React sites from scratch, what use would there really be for Gatsby?

Fortunately, the tutorial for Gatsby is also a great way to gain knowledge of React in general. If you only use Gatsby for your first React app, a few things like auto-link formatting will seem like they’re core React tools when they’re really part of Gatsby. But that shouldn’t be a dealbreaker for anyone.

Having written zero React in the last year, I found my first few sites a cinch with Gatsby after spending a couple hours in their tutorials. If I can do it, anyone can—yes a logical fallacy on the SAT, but true here.

Can we get the team away from Git?

Here we get to the real potential of Gatsby: for our non-git-savvy team members, Gatsby can consume the database of another Content Management System (CMS), seamlessly importing articles with your main site. A CMS can present an input and editor for articles without taking responsibility for displaying the content it stores. Used this way it’s ineptly called a ‘headless CMS’. The good news is, your content contributors can now publish content on their own, without needing to do a code push.

Gatsby is Different

I’m not aware of any other static site generator that has this kind of functionality out of the box. Problems like scheduling posts ahead of time, editing posts (without needing to edit HTML) and content with multiple links and embedded files can all be handled by a tried-and-true editor like WordPress, while Gatsby generates a high-performance React app from the data. Gatsby is worth the plunge. Let me know how it goes for your team!

Chaos Engineering Ideas for Serverless
Danielle Heberling

Danielle Heberling | January 24, 2019

Chaos Engineering Ideas for Serverless

The Principles define chaos engineering as:

The discipline of experimenting on a distributed system in order to build confidence in the system’s capability to withstand turbulent conditions in production.

The high-level steps for implementing chaos experiments involve: defining your application’s steady state, hypothesizing the steady state in both the control and experimental groups, injecting realistic failures, observing the results, and making changes to your code base/infrastructure as necessary based off of the results.

Chaos experiments are not meant to replace unit and integration tests. They’re intended to work with those existing tests in order to assure the system is reliable. A great real-world analogy is that chaos experiments are like vaccines: A vaccine contains a small amount of the live virus that gets injected into the body in order to prompt the body to build up immunity to prevent illness. With chaos experiments, we’re injecting things like latency and errors into our application to see if the application handles them gracefully. If it does not, then we can adjust accordingly in order to prevent incidents from happening.

Sometimes chaos engineering gets a bad reputation as ‘breaking things for fun.’

Sometimes chaos engineering gets a bad reputation as “breaking things for fun.” I believe the problem is that there’s too much emphasis on breaking things while the focus should be on why the experiments are being run. In order to minimize your blast radius, it’s recommended to begin with some experiments in non-production environments during the workday while everyone is around to monitor the system. On the people side of things, make sure you communicate with your entire team what you’re doing, so they aren’t caught by surprise. Once you have experience and confidence running experiments, you can then move onto running them in production. The end goal is to run experiments in production since it is difficult to have an environment that matches production exactly.

Traditionally chaos engineering at a high level is running experiments that often involve shutting off servers, but if you are in a serverless environment with managed servers this can pose a new challenge. Serverless environments typically have smaller units of deployment, but more of them. This means for someone who wants to run chaos experiments, there are more boundaries to harden around in your applications.

If you’re thinking about running some chaos experiments of your own in a serverless environment, some ideas of things to look out for are:

  • Performance/latency (most common)
  • Improperly tuned timeouts
  • Missing error handling
  • Missing fallbacks
  • Missing regional failover (if using multiple regions)

For serverless, the most common experiments involve latency injection or error injection into functions.

Some examples of errors you could inject are:

  • Errors common in your application
  • HTTP 5xx
  • Amazon DynamoDB throughput exceeded
  • Throttled AWS lambda invocations

A pretty neat trend I’ve seen is folks writing their own libraries to inject latency and/or errors into a Lambda function using the new Lambda layers feature. Stackery makes it easy to add layers to your function. Another idea is to implement chaos experiments as part of your CI/CD pipeline. If you don’t want to write your own library there are a lot of open source projects and also some companies that offer “chaos-as-a-service.”

If you’d like to go into more detail on implementation, I’d suggest checking out this article to see some code. This GitHub repo also has some great resources on the overall topic of Chaos Engineering. I hope this post gave you some ideas and inspiration on different ways you can test your serverless environment to ensure system reliability for your customers!

Serverless in 2019: From 'Hello World' to 'Hello Production'
Nate Taggart

Nate Taggart | January 04, 2019

Serverless in 2019: From 'Hello World' to 'Hello Production'

A Look Ahead

As the CEO of Stackery, I have had a unique, inside view of serverless since we launched in 2016. I get to work alongside the world’s leading serverless experts, our customers, and our partners and learn from their discoveries. It’s a new year: the perfect time to take stock of professional progress, accomplishments, and goals. The Stackery team has been in this mindset for months, focusing on what 2019 means for this market. After two-and-a-half years of building serverless applications, speaking at serverless conferences, and running the world’s leading serverless company, I have a few ideas of what’s in store for this technology.

1) Serverless will be “managed cloud services,” not “FaaS”

As recently as a year ago, every serverless conference talk had an obligatory “what is serverless” slide. Everyone seemed to have a different understanding of what it all meant. There were some new concepts, like FaaS and “events” and a lot of confusion on the side. By now, this perplexity has been quelled and the verdict is in: serverless is all about composing software systems from a collection of cloud services. With serverless, you can lean on off-the-shelf cloud services resources for your application architecture, focus on business logic and application needs, while (mostly) ignoring infrastructure capacity and management.

In 2019, this understanding will reach the mainstream. Sure, some will continue to fixate on functions-as-a-service while ignoring all the other services needed to operate an application. Others will attempt to slap the name onto whatever they are pitching to developers. But, for the most part, people will realize that serverless is more than functions because applications are more than code.

I predict that the winners in serverless will continue to be the users capturing velocity gains to build great applications. By eschewing the burden of self-managed infrastructure and instead empowering their engineers to pull ready-to-use services off the shelf, software leaders will quickly stand up production-grade infrastructure. They’ll come to realize that this exciting movement is not really “serverless” so much as it is “service-full” - as in applications full of building blocks as a service. Alas, we’re probably stuck with the name. Misnomers happen when a shift is born out of necessity, without time to be fine-tuned by marketing copywriters. I’ll take it.

2) The IT Industrial Complex will throw shade

The IT Industrial Complex has billions of dollars and tens of thousands of jobs reliant on the old server model. And while these vendors are cloud-washing their businesses, the move to serverless renders them much less excited about the cloud-native disruption.

So get ready for even more fear, uncertainty, and doubt that the infrastructure old-guard is going to bring. It won’t be subtle. You’ll hear about the limitations of serverless (“you can’t run long-lived jobs!”), the difficulty in adoption (“there’s no lift-and-shift!”), and the use cases that don’t fit (“with that latency, you can’t do high-frequency trading!”). They’ll shout about vendor lock-in — of course they’d be much happier if you were still locked-in with their physical boxes. They’ll rail against costs (“At 100% utilization, it’s cheaper to run our hardware”), and they’ll scream about how dumb the name “serverless” is (you’ve probably gathered that I actually agree with this one).

I’d rather write software than patch infrastructure any day.

The reality? The offerings and capabilities of the serverless ecosystem are on an improvement velocity, unlike anything the IT infrastructure market has ever delivered. By the end of 2019, we’ll have more languages, more memory, longer run times, lower latency, and better developer ergonomics. They’ll ignore the operational cost of actually running servers — and patching, and scaling, and load-balancing, and orchestrating, and deploying, and… the list goes on! Crucially, they’ll ignore the fact that every company invested in serverless is able to do more things faster and with less. Serverless means lower spend, less hassle, more productive and focused engineers, apps with business value, and more fun. I’d rather write software than patch infrastructure any day.

Recognize these objections for what they are: the death throes of an out-of-touch generation of technology dinosaurs. And, as much as I like dinosaurs, I don’t take engineering advice from them.

3) Executives will accelerate pioneering serverless heroes

Depending on how far your desk is from the CEO of your company, this will be more or less obvious to you, but: your company doesn’t want to invest in technology because it’s interesting. Good technology investments are fundamentally business investments, designed to drive profits by cutting costs, innovation, or both.

Serverless delivers on both cost efficiency and innovation. Its pay-per-use model is substantially cheaper than the alternatives and its dramatically improved velocity means more business value delivery and less time toiling on thankless tasks. The people who bring this to your organization will be heroes.

So far, most organizations have been adopting serverless from the bottom-up. Individual developers and small teams have brought serverless in to solve a problem and it worked. But in 2019 a shift will happen. Project milestones will start getting hit early, developers will be more connected to customer and business needs, and IT spend will come in a little lower than budgeted… And the executive team is going to try to find out why, so they can do more of it.

So my prediction is that in 2019, serverless adoption will begin to win executive buy-in and be targeted as a core technology initiative. Serverless expertise will be a very good look for your team in 2019.

4) The great monolith to serverless refactoring begins

While greenfield apps led the way in serverless development, this year, word will get out that serverless is the fastest path to refactoring monoliths into microservices. In fact, because serverless teams obtain significant velocity from relying largely on standard infrastructure services, many will experience a cultural reset around what it means to refactor a monolith. It’s easier than ever before.

While “you can’t lift and shift to serverless” was a knock in 2018, 2019 will show the enterprise that it’s faster to refactor in serverless than migrate. They will see how refactoring in serverless takes a fraction of the time we thought it would take for a growing number of applications. Check out the Strangler Pattern to see how our customers are doing this today. When you combine this method with Lambda Layers and the rapid march of service innovations, the options for evolving legacy applications and code continue to broaden the realm of where serverless shines.

5) Serverless-only apps will transition to serverless-first apps

“Hello World” applications in tutorials are good fun and their initial functions deliver rapid purpose without an operations team. They are great wins for serverless.

However, when it comes to building serverless business applications, every software team will need to incorporate existing resources into their applications. Production databases and tables, networks, containers, EC2 instances, DNS services, and more. Today, complex YAML combined with the art of managing parameters across dev, test, staging, and production environments hold many teams back from effectively building on what already exists. A note: Stackery makes using existing resources across multiple environments easy.

In 2019, serverless will serve you more.

In 2019, we’ll see enormous growth in applications that are serverless-first, but not serverless only. The “best service for the job” mantra is already driving teams under pressure to deliver results to serverless. We believe teams who want to move fast will turn to serverless for most of what they need, but won’t live in a serverless silo.

To conclude: In 2019, serverless will serve you more.

All of these predictions add up to one obvious conclusion from my perspective: Serverless is finally mainstream and it’s here to stay. Stackery already helps serverless teams accelerate delivery from “Hello World” to “Hello Production”. We’d love to help your team, too.

Conquering a Double-Barrel Webpack Upgrade
Anna Yovandich

Anna Yovandich | December 20, 2018

Conquering a Double-Barrel Webpack Upgrade

Over the last couple of weeks, we’ve prioritized some sustaining product goals to polish the codebase and update some big ticket dependencies. Among those updates were: React, Redux, and Webpack - the biggies. The first two were pretty painless and inspired the confidence to approach updating Webpack from v2 to v4 like maybe no big deal! Though confidence level was on high, I felt a slight chill and a twinge of doubt by the prospect of making changes to our build configs.

Enter Webpack 4

The latest version of Webpack has the lowest barrier to entry of any other version. Its new mode parameter comes with default environment configs and enables built-in optimizations. This “no config” option is ideal for a new project and/or a newcomer to Webpack that wants to get started quickly. Migrating an existing config is a little trickier but following the migration guide got our development environment in pretty good shape. I was pleasantly shocked by the Webpack documentation. It’s thorough, well organized, and has improved significantly from the early days of v1.

Development Mode

To begin migrating our development config, I added the new mode property, removed some deprecated plugins, and replaced autoprefixer with postcss-preset-env in the post-css-loader plugin config. Starting the dev server (npm start) at this point led to the first snag: this.htmlWebpackPlugin.getHooks is not a function. Hunting that error landed in an issue thread, suggesting a fix - which did the trick. Development mode: good to go. Confidence mode: strong.

Production Mode

Continuing migration with the production config was a similar process. We have a fairly standard setup to compile the static build directory: transpile (ES6 and JSX) and minify JS; transform, externalize, and minify CSS; then generate an index.html file to tie it all together. However, running the production build (npm run build) was a different story.

FATAL ERROR

The first issue was harsh: FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory. Ooof! Lots of searching and skimming repeatedly offered the same suggestion: to pass an argument to the node process --max_old_space_size=<value> which increases the heap memory allocation. It felt like slapping some tape on a shiny new toy but it enabled the build process to complete successfully.

Feeling unsatisfied with band-aiding an ominous failure, I investigated why the build was consistently choking on source map generation and here is where I discovered a 2 alarm fire:

  1. Our main (and only) bundle is 1.6MB
  2. That one giant bundle is accompanied by a behemoth source map…19MB to be exact. 😱 Not ok.
Code Splitting

First, the bundle needs to be split by configuring optimization.splitChunks. Then, the vendor source maps need to be excluded by configuring SourceMapDevToolPlugin exclude option. An important step when using SourceMapDevToolPlugin, is setting dev-tool: false. Otherwise, the its configuration (with exclude rules) will get trampled by Webpack’s dev-tool operation and output another monster source map (mapping the entire build again).

devtool: false,
optimization: {
 splitChunks: {
  chunks: 'all',
  name: true,
  cacheGroups: {
   vendors: {
    test: /[\\/]node_modules[\\/].*\.js$/,
    filename: 'static/js/vendors.[chunkhash:8].js',
    priority: -10
   },
   default: {
    minChunks: 2,
    priority: -20,
    reuseExistingChunk: true
   }
  }
 }
}
...
plugins: [
 new webpack.SourceMapDevToolPlugin({
  filename: 'static/js/[name].[chunkhash:8].js.map',
  exclude: /static\/js\/vendors*(.+?).js/
 })
]

With the build output in much better shape (though the vendors bundle should be further split into smaller chunks), I try removing the node argument band-aid and re-running the build command (sans gargantuan source map). Success! The fatal error was almost exclusively due to source mapping one enormous build.

Minify CSS

Now the build succeeds and I’m cookin with gas. However, the CSS file is much bigger than it used to be…it’s no longer minified. One of the plugins that changed with this upgrade was replacing ExtractTextPlugin with MiniCssExtractPlugin (extracts all css modules into a separate file). However, MiniCssExtractPlugin does not handle minification(https://github.com/webpack-contrib/mini-css-extract-plugin#minimizing-for-production) like ExtractTextPlugin did. To minify CSS, the OptimizeCSSAssetsWebpackPlugin (aka OCAWP) is necessary.

To include OCAWP, add optimization.minimizer configuration to the module:

optimization: {
 minimizer: [
  new OptimizeCSSAssetsWebpackPlugin({
   cssProcessorOptions: {
    parser: require('postcss-safe-parser'),
    map: {
     inline: false,
     annotation: true
    }
   },
   cssProcessorPluginOptions: {
    preset: ['default', {
     discardComments: {
      removeAll: true
     }
    }]
   }
  })
 ]
}

Now, CSS is minified but…JavaScript is not. 😑 Hoo boy.

Minify JS

By default, Webpack uses UglifyJs to minify JavaScript. When optimization.minimizer is customized (in this case for CSS minification), JS minification needs to be explicitly handled as well. Now the optimization.minimizer config contains OCAWP and UglifyJs but the build script fails again - citing: Unexpected token: keyword (const) error from UglifyJs. Siiigh.

It turns out, uglify-js (the parser used by UglifyJsWebpackPlugin) does not support ES6 uglification. The maintainer of UglifyJsWebpackPlugin, as well as the Webpack docs urge the adoption of TerserWebpackPlugin instead. This works out great, since the next version of Webpack will use Terser as its default minifier. Thank you, next!

optimization: {
 minimizer: [
  new OptimizeCSSAssetsWebpackPlugin({...}),
  new TerserWebpackPlugin({
   sourceMap: true,
   parallel: true,
   terserOptions: {
    parse: {
     ecma: 8
    },
    compress: {
     ecma: 5,
     warnings: false,
     comparisons: false,
     inline: 2
    },
    output: {
     ecma: 5,
     comments: false,
     ascii_only: true
    }
   }
  })
 ]
}

The production build is finally compiling as expected. There are still improvements to be made but I will rest easier knowing that this configuration isn’t exploding CPUs and that I have a better grip on optimizations going forward.

It’s been a tough and humbling week. Configuring Webpack’s loaders and plugins correctly can feel overwhelming - there are countless options and optimizations to understand. If you or someone you love is going through frontend dependency hardships, just know: it gets better and you are not alone. Hang in there!

Webhooks Made Easy with Stackery
Anna Spysz

Anna Spysz | November 08, 2018

Webhooks Made Easy with Stackery

Webhooks are about as close as you can get to the perfect serverless use case. They are event-driven and generally perform just one (stateless) function. So of course we wanted to show how easy it is to implement webhooks using Stackery.

Our newest tutorial, the Serverless Webhooks Tutorial, teaches you to create a GitHub webhook and connect it to a Lambda function through an API Gateway. Or, to put it simple terms: when your GitHub repository does a thing, your function does another thing. What that second thing does is completely up to you.

Here are some possible use cases of a GitHub webhook:

  • Connect your webhook to the Slack API and have Slack ping your team members when someone has opened a PR
  • Have your function deploy another stack when its master branch is updated
  • Expanding on that, you can even have your function deploy to multiple environments depending on which branch has been updated
  • Write an Alexa Skill that plays a certain song when your repository has been starred - the possibilities are endless!

The best part is, GitHub allows you to be very specific in what events you subscribe to, and you can further narrow down events in your function logic.

So for example, do you want to be notified by text message every time Jim pushes a change to the master branch of your repository, because Jim has been known to push buggy code? You can set that up using webhooks and Stackery, and never have master go down again (sorry, Jim).

Check out the tutorial to see what else you can build!

Building a Single-Page App With Stackery & React
Jun Fritz

Jun Fritz | October 30, 2018

Building a Single-Page App With Stackery & React

After completing this tutorial, you’ll have a serverless SPA built using Stackery and React. Stackery will be used to configure, deploy, and host our application which will be built using the React library.

The newest tutorial on our documentation site guides you through the process of building a Serverless Single-Page App using Stackery and React.

You’ll be using Stackery to set up the cloud resources needed to deploy, host, and distribute your single-page application. You’ll configure a Lambda function, an S3 Bucket, and a CloudFront CDN in this tutorial with the goal of keeping this application within AWS Free Tier limits.

By the end of this tutorial, you’ll have a fully-scalable backend and an organized React front-end to add to, and grow your application. Watch part one of the tutorial below to see what we’re building, or follow along with the plain-text version here.

Stay tuned for more serverless tutorials from Stackery!

Building Serverless Applications with AWS Amplify
Danielle Heberling

Danielle Heberling | October 24, 2018

Building Serverless Applications with AWS Amplify

So you want to use AWS Cognito to authenticate users and have your user pool, identity pool, and app client all set up in the AWS console. …the next question is how can you connect this with your React based frontend? While there are a few ways to go about doing this, this post is going to give you a brief overview on how to do this via a library called AWS-Amplify.

AWS-Amplify is an open source project managed by AWS described as “a declarative JavaScript library for application development using cloud services.” I liked this particular library, because it has a client first approach and abstracts away some of the setup required in the JavaScript SDK.

My favorite features of Amplify are: Authentication (via Cognito), API (via API Gateway), and Storage (via S3), but this library has a lot more to offer than just those features. This post will focus on how to authenticate users from a React based frontend…more specifically user signup that has an email address verification step.

The Setup

First you’ll need to setup a config file to reference your already created AWS resources (in this case the user pool, identity pool, and client id) in your /src folder. The file will look something like this :

src/config.js

export default {
   cognito: {
    REGION: ‘YOUR_COGNITO_REGION’,
    USER_POOL_ID: ‘YOUR_USER_POOL_ID’,
    APP_CLIENT_ID: ‘YOUR_APP_CLIENT_ID’,
    IDENTITY_POOL_ID: ‘YOUR_IDENTITY_POOL_ID’
  }
};

Then in your index.js file where you setup your react app, you’ll need to configure aws Amplify. It’ll look similar to this:

src/index.js

import React from ‘react’;
import ReactDOM from ‘react-dom’;
import Amplify from ‘aws-amplify’;

import config from ‘./config’;
import App from ‘./App’;

Amplify.configure({
  Auth: {
    mandatorySignIn: true,
    region: config.cognito.REGION,
    userPoolId: config.cognito.USER_POOL_ID,
    identityPoolId: config.cognito.IDENTITY_POOL_ID,
    userPoolWebClientId: config.cognito.APP_CLIENT_ID
  }
});

ReactDOM.render(
  <Router>
    <App />
  </Router>,
  document.getElementById(‘root’)
);

The mandatorySignIn property is optional, but is a good idea if you are using other AWS resources via Amplify and want to enforce user authentication before accessing those resources.

Also note that for now having a separate config file might seem a bit overkill, but once you add in multiple resources (i.e. Storage, API, Pub Sub etc.) you’ll want that extra config file to keep things easy to manage.

Implementation Overview

The signup flow will look like this:

  1. The user submits what they’ll use for login credentials (in this case email and password) via a signup form and a second form to type in a confirmation code will appear.
  2. Behind the scenes the Amplify library will sign the user up in Cognito.
  3. Cognito will send a confirmation code email to the user’s signup email address to verify that the email address is real.
  4. The user will check their email > get the code > type the code into the confirmation form.
  5. On submit, Amplify will send the information to Cognito which then confirms the signup. On successful confirmation, Amplify will sign the user into the application.

Implementation Part 1

First in your signup form component, you’ll need to import Auth from the Amplify library like this:

import { Auth } from ‘aws-amplify’;

As you create your form, I’d suggest using local component state to store the form data. It’ll look like your typical form with the difference being using the Amplify methods in your handleSubmit function whenever the user submits the form. The handleSubmit function will look like this:

 handleSubmit = async event => {
    event.preventDefault();

    try {
      const newUser = await Auth.signUp({
        username: this.state.email,
        password: this.state.password
      });
      this.setState({
        newUser
      });
    } catch (event) {
      if (event.code === ‘UsernameExistsException’) {
        const tryAgain = await Auth.resendSignUp(this.state.email);
        this.setState({
          newUser: tryAgain
        });
      } else {
        alert(event.message);
      }
    }
  }

On success, Amplify returns a user object after the signUp method is called, so I’ve decided to store this object in my component local state so the component knows which form to render (the signup or the confirmation).

Before we continue let’s go over a quick edge case. So if our user refreshes the page when on the confirmation form and then tries to sign up again with the same email address, they’ll receive an error that the user already exists and will need to signup with a different email address. The catch block demonstrates one way of handling that possibility by resending the signup code to the user if that email is already present in Cognito. This will allow the user to continue using the same email address should they refresh the page or leave the site before entering the confirmation code.

Implementation Part 2

So now the user is looking at the confirmation form and has their confirmation code to type in. We’ll need to render the confirmation form. Similar to the signup form it’ll look like a typical form with the exception being the function that is called whenever the user submits the confirmation form. The handleSubmit function for the confirmation form will look similar to this when using Amplify:

 handleConfirmationSubmit = async event => {
    event.preventDefault();

    try {
      await Auth.confirmSignUp(this.state.email, this.state.confirmationCode);
      await Auth.signIn(this.state.email, this.state.password);

      this.props.isAuthenticated(true);
      this.props.history.push("/");
    } catch (event) {
      alert(event.message);
    }
  }

So it is taking in the form data, using Amplify to confirm the user’s email address via the conformation code and signing in the user if successful. You can then verify if a user is signed in via props at the route level if you’d like. In this case, I arbitrarily named it isAuthenticated and redirected the user to the root path.

The complete docs for using the Auth feature of Amplify can be found here. We’ve only scratched the surface in this post, so go forth and explore the all of the different features that Amplify has to offer. I’ve found it has a very nice declarative syntax and is very readable for folks who are new to a codebase. For building further on your React-based serverless applications, I highly recommend Stackery for managing all of your serverless infrastructure backed up by seamless, git-based version control.

Get the Serverless Development Toolkit for Teams

now and get started for free. Contact one of our product experts to get started building amazing serverless applications today.

To Top