Stacks on Stacks

If you’ve been over to our documentation site lately, you may have noticed some changes. We’ve got a new look and some new tutorials, but the latest upgrade is our new Quickstart tutorial.

While the first version of our Quickstart just got you up and running with Stackery, version 2.0 also has you deploying a static HTML portfolio page to an API endpoint:

Oooh, fancy!

Once you’ve followed the tutorial and deployed your static site, you can customize the HTML with your own information and links to your projects. You can then follow our serverless contact form tutorial to give the contact form on your site functionality as well.

Want a preview? This YouTube video walks you through the entire Quickstart tutorial:

And be sure to visit our docs site regularly, as we have several new tutorials in the works. Stay tuned for a React application with a serverless backend - coming soon!

Last month, our CTO Chase wrote about why you should stop YAML engineering. I completely agree with his thesis, though for slightly different reasons. As a new developer, I’ve grasped that it’s crucial to learn and do just what you need and nothing more - at least when you’re just getting started in your career.

Now, I’m all about learning for learning’s sake - I have two now-useless liberal arts degrees that prove that. However, when it comes to being a new developer, it’s very easy to get overwhelmed by all of the languages and frameworks out there, and get caught in paralysis as you jump from tutorial to tutorial and end up not learning anything very well. I’ve certainly been there - and then I decided to just get good at the tools I’m actually using for work, and learn everything else as I need it.

Which is what brings us to YAML - short for “YAML Ain’t Markup Language”. I started out as a Python developer. When I needed to, I learned JavaScript. When my JavaScript needed some support, I learned a couple of front-end frameworks, and as much Node.js as I needed to write and understand what my serverless functions were doing. As I got deeper into serverless architecture, it seemed like learning YAML was the next step - but if it didn’t have to be, why learn it? If I can produce 200+ lines of working YAML without actually writing a single line of it, in much less time than it would take me to write it myself (not counting the hours it would take to learn a new markup language), then that seems like the obvious solution.

So if a tool allows me to develop serverless apps without having to learn YAML, I’m all for that. Luckily, that’s exactly what Stackery does, as you can see in the video below:

Amazon Web Services SAM is a developer-centric, cloud native, open source framework to define serverless applications faster and with better consistency. SAM provides a standard for defining the architecture of serverless projects. Stackery now supports this framework natively and provides the tools for maintaining best practices, streamlining workflows, and enforcing consistency.

Amazon continues to move the industry forward from static, monolithic technology to a much more distributed landscape. Because it’s an open source standard heavily supported by and aligned with AWS, SAM benefits from Amazon’s prominence in the industry and the ecosystem established around them. As a part of Amazon’s toolbox, SAM comes with the resources and support of SAR, the Serverless Application Repository.

Velocity and Efficiency

So what changes are expected from Stackery now that it deploys to AWS using SAM? On the surface it might not look like much of anything has changed. Stackery customers still have full control and everything will continue to be stored and accessible in the same AWS account.

SAM is a open source community supported standard, heavily backed by AWS, the clear leader for hosting serverless apps. Opting for SAM means that you’re building your apps on the leading open source standard with a vibrant community to turn to when you need support, and it gives you to access to all the tools and advantages of such an ecosystem, like SAM local development mode.

Stackery endows you with the power to generate and manage SAM templates directly. Maintain full control while increasing consistency and efficiency by automating the most repetitive infrastructure configuration tasks. Just a few clicks in the Stackery Operations Console can generate new infrastructure-as-code configurations — You can also manually edit existing SAM templates and alter them to fit your needs. Doing that any other way would require significantly higher time and work investments.

Consistency and Scalability

Move fast without the fear that you’ll make mistakes. Engineering time is better spent working on core problems than small, monotonous tasks but the nature of writing software means consistency and attention to detail are vital. Every organization wants to move fast — leadership and engineering teams can always agree on that much — and Stackery with SAM enables higher velocity without sacrificing consistency. By building with Stackery on SAM, developers can enjoy the benefits of Lambda’s scalability while consistently shipping stable code and minimizing the serverless learning curve.

At Stackery, we want people to be able to use their own tools and write their applications the way they want. For developers who want to move fast, SAM makes that possible without creating new abstractions that cause trouble in the long run. Move fast with stable infrastructure.

Stackery is excited to announce the company has achieved Advanced Tier Technology Partner status in the Amazon Web Services (AWS) Partner Network (APN). Stackery has achieved this recognition by meeting strict quality and accreditation standards for Amazon Web Services partners.

What This Means for Stackery

Stackery has always been dedicated to providing the best platform for building serverless applications that utilize some of the most advanced capabilities that AWS has to offer. Stackery underwent a rigorous architectural review by AWS and was found to meet all of the the technical requirements for APN Advanced Technology Partners. This new partner status validates our commitment to building a great product that meets all of the standards and best practices that the world’s leading cloud provider requires.

What This Means for Stackery Customers

As a part of the approval process, Stackery had to meet Amazon’s strict requirements for security, site reliability, disaster recovery and customer data protection. This means that Stackery customers have the assurance that their serverless architecture is protected and maintained according to AWS standards.

“As we move forward with improving and expanding our product, it is important that we stay committed to giving our customers the best data protection and reliability possible.” said Nate Taggart, CEO of Stackery. “We are excited about our new relationship with Amazon and look forward to bringing more tools to our customers that will help their businesses grow in the cloud.”

Our product engineering team ships every single day.

That means Stackery’s product gets better every single day. Stackery engineers commit code into git which marches into our continuous delivery pipeline. We promote each version of our microservices, frontend, and CLI through multiple testing environments, rolling shiny new features into production or notifying the team of failures. This is the best way we know to develop modern software and explains why our team is able to ship so much functionality so rapidly.

However, because we’re constantly shipping, it means we need to pause periodically to take note of new features and improvements. In this post I’ll summarize some of the most significant features and changes from our product team over the past few months. For a more detailed list of changes, you can read and/or follow Stackery’s Release Notes.

Referenced Resource

One of the best things about microservice architecture is the degree which you can encapsulate and reuse functionality. For example, if you need to check if a user is authorized to perform a certain action, there’s no need to scatter permissioning code throughout your services. Put it all in one place (an AuthorizationService perhaps), and call out to that in each service that needs to check permissions.

Stackery’s Referenced Resource nodes let’s you reference existing infrastructure resources (be they Lambda functions, S3 buckets, VPCs, you name it) by their AWS ARN and seamlessly integrate these into your other services.

One of the best uses I’ve seen for Referenced Resources is using it as the mechanism to implement centralized error reporting for serverless architectures. Write one central Lambda function that forwards exceptions into your primary error reporting and alerting tool. Configure every other stack to send error events to this central handler. Viola! Complete visiblity into all serverless application errors.

Support for Multiple AWS Accounts

Every company we work with uses multiple AWS accounts. Sometimes there’s one for production and one for everything else. In Stackery’s engineering team each engineer has multiple accounts for development and testing, as well as shared access to accounts for integration testing, staging, and production. Splitting your infrastructure across multiple accounts has major benefits. You can isolate permissions and account-wide limits, minimizing risk to critical accounts (e.g. production).

However, managing deployment of serverless architectures across multiple accounts is often a major PITA. This is why working across multiple accounts is now treated as a first class concern across all of Stackery’s functionality. Multiple AWS accounts can be registered within a Stackery account using our CLI tool. Stackery environments are tied to an AWS accounts, which maps flexibly into the vast majority of AWS account usage patterns.

Managing multiple AWS accounts is a key part of most organizations’ cloud security strategy. Stackery supports this by relying on your existing AWS IAM policies and roles when executing changes. If the individual executing the change doesn’t have permission in that AWS account, the action will fail. This makes it straightforward to set up workflows where engineers have full control to make changes in development and testing environments, but can only propose changes in the production account, which are then reviewed and executed by an authorized individual or automation tool.

You can read more in our knowledge base article about Working with multiple AWS accounts in Stackery

CloudFormation Resource Nodes

Sometimes you need to do something a little different, which is why we built custom CloudFormation Resource nodes. You can use these to provision any AWS resource and take advantage of the full power and flexibility of CloudFormation, for situations when that’s required or desireable.

What’s been coolest about rolling this feature out is the variety of creative uses we’ve seen it used. For example use CloudFormation Resource nodes to automatically configure and seed a database the first time you deploy to a new environment. You can also use them to automatically deploy an HTML front end to CloudFront each time you deploy your backend serverless app. The possibilities are endless.

AWS Resource Tagging

Resource Tagging may not be the most glamorous of features, but it’s a critical part of most organizations’ strategies for tracking cost, compliance, and ownership across their infrastructure. Stackery now boasts first class support for tagging provisioned resources. We also provide the ability to require specific tags prior to deployment, making it orders of magnitude to get everyone on the same page on how to correctly tag resources.

Always Shipping

Our goal is to always be shipping. We aim to push out valuable changes every day. Customer’s gain more control and visiblity over their serverless applications each day, so they can ship faster and more frequently too. Look out for more great changes rolling out each day in the product, and watch this blog for regular announcements summarizing our progress. We also love to hear what you think so if you have wants or needs managing your serverless infrastructure, don’t hesitate to let us know.

I’ve always wanted to build a Reddit bot, however, I didn’t want to go through the hassel of actually setting up cloud based hosting for it to run on. One of the most powerful aspects of serverless architectures is how simple it is to implement a task pipeline. In this case, I created a fully live Reddit bot in about an hour, that scrapes the top posts from /r/cooking and emails them to me. It’s easy to see how these atomic types of tasks can be chained together to create powerful applications. For example, with a bit more work, instead of an AWS SNS topic we could feed the Reddit posts into an AWS Kinesis Stream, then attach consumer lambda functions to the stream to perform context analytics. One can see how this can apply to a CI/CD pipeline, and in fact we use similar processes with our own serverless continuous integration (CI) and continuous delivery (CD) pipeline. Read more about Stackery’s CI/CD here.

Overview of Components

• “Timer” node to ping a function, triggering the reddit bot to work once a day
• “RedditBot” node, a lambda function that once triggered, authenticates with reddit using the snoowrap library and scrapes the hot /r/cooking posts, sending along the good ones via SNS
• “HotCookingPosts” SNS topic node, an SNS topic that forwards all messages to my email address

Implementation Details

Create a reddit account for your bot. Then navigate to https://www.reddit.com/prefs/apps/ and select Create App, making sure you select “script” in the radios underneath the name. Note down the client id and client secret, these will go into the function configuration along with the reddit username and password for your bot account.

Configure a stack using the Stackery dashboard with 3 nodes:

Timer -> Function -> SNS Topic

Attached to the function are some configuration values that are necessary for reddit’s authentication mechanism. Stackery automatically includes certain information about a function based on what it’s attached to (in this case, the SNS topic). Read more about the output port data here. We can leverage this when specifying the topic node ARN for forwarding on the selected posts, implemented in this file.

Function Settings:

Configuration Environment Variables:

Fill in your saved client id, secret, Reddit bot username, and Reddit bot password and store under environment variable names in the function editor panel. For more information on how to create a deployment environment configuration, visit the Environment Configuration Docs. It’s important not to add these sensitive variables directly, as they will then be committed to github, and (depending on your repository settings) exposed to the public. When added via an environment configuration, these key value pairs are automatically encrypted and stored in an S3 bucket on your AWS account.

The “bot” function will receive a timer event which then triggers it to scrape /r/cooking. The timer interval can be triggered every minute to test functionality, then I’d recommend changing it to a more sane interval.

The function looks through the hot submissions and any with > 50 comments get forwarded to the SNS topic. See the code for this here: https://github.com/stackery/stackery-reddit-bot

You can also insert log statements in your own code to debug the lambda function via Cloudwatch Logs (which you can easily get to the logs in from the function’s metrics tab section in the Stackery deployments view).

Currently, the code sends a json object directly to email. This is done by navigating to your AWS accounts SNS service, to the topic that Stackery has automatically provisioned, and clicking the Create Subscription button, with the Protocol field set to email, and value as your email. For more on the capabilities of SNS visit Amazon’s SNS Docs.

As you can see, it’s really straightforward to build a Reddit bot (and many other types of bots) using serverless resources and Stackery’s cloud management capabilities. Bots are functionally lightweight by nature, and fit easily into serverless architectures. With the composability of AWS Lambda, they can be orchestrated and chained together to perform a variety of tasks, ranging from emailing scraped posts off Reddit, to managing CI/CD.

Reviewing changes in a UI as the result of a pull request is a common occurrence in a development team. This typically involves switching the local working branch to the PR branch, compiling a build, viewing it on localhost, then giving functional/behavioral/visual feedback. There are certainly many solutions to alleviate this context and code switch. One we have built and adopted recently uses Stackery as a CI tool to clone, compile, and preview a pull request.

Check out our guide that details how we built it with step-by-step instruction and sample code.

Developing functions for serverless can get pretty tedious. While there are some solutions to developing a function locally, at this time, they are generally limited in scope and capability and all too often you will find yourself needing to iterate in a real stack. This can end up being quite slow, involving a number of steps that each take time, but not enough to be able to switch off onto another task - develop, commit, prepare, deploy, test, repeat. Some of these steps take time that is somewhat unavoidable, but as this is a problem we face internally at Stackery often, we’ve worked on trying to reduce this with our Stackery CLI tooling.

Stackery enables you to easily design, connect and deploy complicated architectures with an intuitive UI console. However, we recognize that requiring users to switch back and forth from a code editor to a browser to develop, deploy and monitor can be jarring and distracting. That’s why users also have the ability to deploy using Stackery CLI.

Many users have probably only ever used the Stackery CLI when first getting started with Stackery (If this is you or if you haven’t updated the Stackery CLI in a while, you should first run stackery update) . However, the Stackery CLI also lets you easily deploy your stack - just provide it the stack name, environment to deploy into, and a branch name, tag, or commit SHA.

For example, stackery deploy flightTracker dev advancedSearch will deploy the head of the advancedSearch branch of your stack called flightTracker into your dev environment. If you use an editor that has an inline terminal, you can edit, commit, push, deploy and monitor the deployment without ever needing to switch apps. The Stackery CLI also provides the status of the deployment so there’s no longer any need to keep refreshing the AWS CloudFormation console to monitor the status of the deployment.

We’ve been using this capability internally for a while and we’re hoping that you find it as useful as we have. The Stackery CLI also has a number of flags that enable deploying stacks without user input, so we’ve integrated it into our internal CI/CD pipeline, but thats a topic for another post.