Scraping off the Dust: Redeploy of my Rust web app

I do a redeploy of my Rust web app with the Ubuntu 18.04 image on AWS.

Part of a Series: Designing a Full-Featured WebApp with Rust
Part 1: Piecing Together a Rust Web Application
Part 2: My Next Step in Rust Web Application Dev
Part 3: It’s Not a Web Application Without a Database
Part 4: Better Logging for the Web Application
Part 5: Rust Web App Session Management with AWS
Part 6: OAuth Requests, APIs, Diesel, and Sessions
Part 7: Scraping off the Dust: Redeploy of my Rust web app
Part 8: Giving My App Secrets to the AWS SecretManager

Life hasBusy as a bee

Life has been busy – no apologies or excuses, but, ya know, it’s 2020. Yet, I’m trying to slowly make my way back into playing with Rust. I decided to move my EC2 instance from AWS-Linux to the Ubuntu image; for one, I got tired of fighting with LetsEncode to get it to renew my SSL cert every 3 months. Also, I wanted to see how a redeploy of my Rust web app would go and if it still worked (why wouldn’t it?). So, lets see how tough it is to get my environment back to the same place. I took some notes (in case I needed to restart, sigh), so let’s go through it.

Go back to Part 1 to see what this fake web app is about and how I got here – I need to reread it myself! So, first, this is what I ended up needing to add to what I got from the default Ubuntu image:

sudo apt install build-essential checkinstall zlib1g-dev pig-config libssl-dev libpq-dev postgresql postgresql-contrib -y

Lots of that was needed in order to get OpenSSL installed, I was following along hints here. Continuing those instructions, I did:

cd /usr/local/src/
sudo wget https://www.openssl.org/source/openssl-1.1.1g.tar.gz
sudo tar -xf openssl-1.1.1g.tar.gz

cd openssl-1.1.1g
sudo ./config --prefix=/usr/local/ssl --openssldir=/usr/local/ssl shared zlib
sudo make
sudo make test
sudo make install
sudo echo "/usr/local/ssl/lib" > /etc/ld.so.conf.d/openssl-1.1.1g.conf
sudo ldconfig -v
sudo mv /usr/bin/c_rehash /usr/bin/c_rehash.backup
sudo mv /usr/bin/openssl /usr/bin/openssl.backup
sudo nano /etc/environment # to add "/usr/local/ssl/bin" to the PATH

Next, instead of solely storing my code on a potentially tenuous EC2 server, I wanted to keep it backed up on my Google Drive (or whatever you like, this solution works with MANY network storage). I used rclone for my Raspberry Pi photo frame so I was familiar with that already. This is weird though, I don’t really need this for projects I store in GitHub… gotta think about it… maybe I just need a /gdrive synced dir for “things”.

curl https://rclone.org/install.sh | sudo bash
rclone config # to add google drive and authorize it

mkdir ~/projects
mkdir ~/projects/rust



Ok, the most fun step!!

curl https://sh.rustup.rs -sSf | sh
cd ~/projects/rust
git clone git@github.com:jculverhouse/pinpoint_shooting.git

I need nginx for my app

sudo apt install nginx
sudo service nginx start

And now the much more reliable LetsEncrypt using Ubuntu 18.04

# follow instructions at https://certbot.eff.org/lets-encrypt
# setup root cronjob to renew once/week

For my Rocket-powered Rust app, I followed some reminders here to connect it to nginx. Simple enough, really. What’s mostly relevant:

...
server_name pinpointshooting.com; # managed by Certbot
location / {
    proxy_pass http://127.0.0.1:3000;
}
...

What? Nginx still has TLS 1 and 1.1 turned on by default? Followed this and removed those, tested the config, and restarted nginx. All of that I checked with SSLLabs via https://www.ssllabs.com/ssltest/analyze.html :

sudo nano /etc/nginx/nginx.conf # to remove TLS1 TLS1.1 from any line
sudo nano /etc/letsencrypt/options-ssl-nginx.conf # to remove TLS1 TLS1.1 from any line
sudo nginx -t
sudo service nginx reload

I’ll need Postgres for my PinpointShooting app as well, found some steps to follow here, plus I needed to setup for my own app and run the initial migrations to get it up-to-date. That involved another change so I could login with the password from a non-user-account.

cargo install diesel_cli --no-default-features --features posters 
psql -d postgres -U postgres
  create user pinpoint password 'yeahrightgetyourown'; # save this in file .env in app dir
  create database pinpoint;
  grant all privileges on database pinpoint to pinpoint;

sudo nano /etc/postgresql/10/main/pg_hba.conf # to edit "local all all" line to be md5 instead of peer

sudo service postgresql restart
psql -d postgres -U pinpoint # to test password above; just exit

cd ~/projects/rust/pinpointshooting
diesel migration run

Finally:

rustup default nightly # because Rocket, sigh...
cargo update
cargo build --release
target/release/pps &

And, we’re back online! Turns out, a redeploy of my Rust web app was about as easy as I could expect! If the app happens to be running, check it out here (though, there isn’t much to see or anything to do): pinpointshooting.com. Also, browse the repo and feel free to send me comments on how to be better about using idiomatic Rust!

Rust Web App Session Management with AWS

Even more libraries added to my web app as I introduce Amazon’s DynamoDB to handle my session storage

Part of a Series: Designing a Full-Featured WebApp with Rust
Part 1: Piecing Together a Rust Web Application
Part 2: My Next Step in Rust Web Application Dev
Part 3: It’s Not a Web Application Without a Database
Part 4: Better Logging for the Web Application
Part 5: Rust Web App Session Management with AWS
Part 6: OAuth Requests, APIs, Diesel, and Sessions
Part 7: Scraping off the Dust: Redeploy of my Rust web app
Part 8: Giving My App Secrets to the AWS SecretManager

Printed security pass - an old school session management practice
Didn’t someone mention a cookie too!? Where’s my cookie!?

I’ve worked on some more bits with my pretend web application I introduced a few posts ago. Thinking about the technology needed to let users register and later log in, session management seems like a good first step to take. I would like to store the sessions in Amazon’s DynamoDB too, just for fun and experience. I’ve pushed several commits to the repository lately, so let me go through some of the changes enabling Rust web app session management with AWS DynamoDB.

First, I searched and read up a bit on best practices for session management. I found a Session Management Cheat Sheet from OWASP. Also, more generically, 12 Best Practices for User Account, Authorization, and Password Management on the Google Cloud Blog. I haven’t addressed many of these yet, but I’ll come back to resources like this several times in the future. I was trying to find language-independent whitepapers or guides on web app session management… and something not a decade old. If anyone has a definitive resource for this, please share!

Rusoto to the Rescue

First, let’s get our web application interfacing with AWS. Rusoto is a big set of crates for using AWS services. Rusoto Core (of course) and DynamoDB are all I need for now, but I am sure I’ll come back for more! To get set up, I created a sessions table in my AWS console and created a programmatic IAM role called pinpointshooting which has full access to the table. When creating a programmatic role, you are given credential access keys which I put into my .env file at the root of the pinpointshooting project. First, connecting to my AWS account with those access keys is a simple matter of:

DynamoDbClient::new(Region::UsEast1)

I had to think about the process I would use to store the session id in a cookie, and when/how it gets created, verified, updated, and deleted. I came up with a simple sketch to make it more clear in my mind. Note that I mention a cache both in my sketch and in some of the comments, but I haven’t done anything about that yet.

  • When a user arrives with no cookie – we need to create a session with some appropriate defaults, write it to the db, and send the cookie in the resulting response.
  • If the user arrives with a cookie – we need to pull the session from the DB (if it really exists) and do some verification and expiration checking. If everything looks good, we pull the session details into our struct. Otherwise, we delete that invalid session data and create a new session like above.
  • When the user selects to log out – we again check to make sure the session is valid, but then delete the session, delete the cookie, and log the user out.



Here are those steps above, as I have them in code at this point!

Step 1: Search for or Create the Session

// Check for sessid cookie and verify session or create new
// session to use - either way, return the session struct
pub fn get_or_setup_session(cookies: &mut Cookies) -> Session {
    let applogger = &LOGGING.logger;
    let dynamodb = connect_dynamodb();

    // if we can pull sessid from a cookie and validate it,
    // pull session from cache or from storage and return
    if let Some(cookie) = cookies.get_private("sessid") {
        debug!(applogger, "Cookie found, verifying";
            "sessid" => cookie.value());

        // verify from dynamodb, update session with last-access if good
        if let Some(mut session) = verify_session_in_ddb(&dynamodb, &cookie.value().to_string()) {
            save_session_to_ddb(&dynamodb, &mut session);
            return session;
        }
    }

    // otherwise, start a new, empty session to use for this user
    let mut hasher = Sha256::new();
    let randstr: String = thread_rng()
        .sample_iter(&Alphanumeric)
        .take(256)
        .collect();
    hasher.input(randstr);
    let sessid = format!("{:x}", hasher.result());

    cookies.add_private(Cookie::new("sessid", sessid.clone()));

    let mut session = Session {
        sessid: sessid.clone(),
        ..Default::default()
    };

    save_session_to_ddb(&dynamodb, &mut session);
    session
}

This will check the (private) cookie sessid, if it came along with the http request, and try to fetch it from DynamoDB and then verify it (see below). If that succeeds, we update the session (with an updated last-access timestamp) and return the session struct. Otherwise, we create a new session id, with some appropriate defaults for now, by getting a hash of a random string of characters. Hrm, I should make sure I didn’t just happen to intersect with an existing sessid at this point! Anyway, we then just add (or update) the sessid cookie to be returned with the http response. Since we are using the private cookie feature of Rocket, the value is actually encrypted. The user can’t see what their real session id is or try to manufacture one. Lastly, if we just created a new session, we save it to the session table and return it.

Step 2: Verify a Session

// Search for sessid in dynamodb and verify session if found
// including to see if it has expired
fn verify_session_in_ddb(dynamodb: &DynamoDbClient, sessid: &String) -> Option<Session> {
    let applogger = &LOGGING.logger;

    let av = AttributeValue {
        s: Some(sessid.clone()),
        ..Default::default()
    };

    let mut key = HashMap::new();
    key.insert("sessid".to_string(), av);

    let get_item_input = GetItemInput {
        table_name: "session".to_string(),
        key: key,
        ..Default::default()
    };

    match dynamodb.get_item(get_item_input).sync() {
        Ok(item_output) => match item_output.item {
            Some(item) => match item.get("session") {
                Some(session) => match &session.s {
                    Some(string) => {
                        let session: Session = serde_json::from_str(&string).unwrap();
                        match session.last_access {
                            Some(last) => {
                                if last > Utc::now() - Duration::minutes(CONFIG.sessions.expire) {
                                    Some(session)
                                } else {
                                    debug!(applogger, "Session expired"; "sessid" => sessid);
                                    delete_session_in_ddb(dynamodb, sessid);
                                    None
                                }
                            }
                            None => {
                                debug!(applogger, "'last_access' is blank for stored session"; "sessid" => sessid);
                                delete_session_in_ddb(dynamodb, sessid);
                                None
                            }
                        }
                    }
                    None => {
                        debug!(applogger, "'session' attribute is empty for stored session"; "sessid" => sessid);
                        delete_session_in_ddb(dynamodb, sessid);
                        None
                    }
                },
                None => {
                    debug!(applogger, "No 'session' attribute found for stored session"; "sessid" => sessid);
                    delete_session_in_ddb(dynamodb, sessid);
                    None
                }
            },
            None => {
                debug!(applogger, "Session not found in dynamodb"; "sessid" => sessid);
                None
            }
        },
        Err(e) => {
            crit!(applogger, "Error in dynamodb"; "err" => e.to_string());
            panic!("Error in dynamodb: {}", e.to_string());
        }
    }
}

That’s some deep nesting – I’m still a newbie at Rust coding, so there is probably a better way to write this. If I didn’t care about logging the problems, I could probably shorten this by using the “?” operator at the end of each step. Maybe I’ll change that eventually, but for now I need to debugging logs to see that things are working.

This chain tries to retrieve the session attribute associated with the sessid (session id encoded in the cookie) and for now, just makes sure it isn’t too old. If that simple check is ok, we deserialize and return that Some(session struct). In all other cases, we return None and a new session will be generated. For cases where a session was present, but determined to be invalid or expired, we also delete the session from DynamoDB.

Step 3: Save (or Update) the Session

// Write current session to dynamodb, update last-access date/time too
fn save_session_to_ddb(dynamodb: &DynamoDbClient, session: &mut Session) {
    let applogger = &LOGGING.logger;

    session.last_access = Some(Utc::now());

    let sessid_av = AttributeValue {
        s: Some(session.sessid.clone()),
        ..Default::default()
    };
    let session_av = AttributeValue {
        s: Some(serde_json::to_string(&session).unwrap()),
        ..Default::default()
    };
    let mut item = HashMap::new();
    item.insert("sessid".to_string(), sessid_av);
    item.insert("session".to_string(), session_av);

    let put_item_input = PutItemInput {
        table_name: "session".to_string(),
        item: item,
        ..Default::default()
    };

    match dynamodb.put_item(put_item_input).sync() {
        Ok(_) => {}
        Err(e) => {
            crit!(applogger, "Error in dynamodb"; "err" => e.to_string());
            panic!("Error in dynamodb: {}", e.to_string());
        }
    };
}

Here, we simply take the session data we are passed, update the last_access to now() and then write the serialized session struct to DynamoDB. Also note, I’m storing the session id inside the session data itself. I’m not sure yet if this is handy duplication, a security concern, or irrelevant.

Step 4: Clean-up After Ourselves

Repairman cleaning up his work, but did anyone check his security badge!?
Hey! Are you a security concern??

Just for completeness, here is the function to drop a session from DynamoDB once we have determined it is invalid (expired).

// Delete session from dynamodb
fn delete_session_in_ddb(dynamodb: &DynamoDbClient, sessid: &String) {
    let applogger = &LOGGING.logger;

    let av = AttributeValue {
        s: Some(sessid.clone()),
        ..Default::default()
    };

    let mut key = HashMap::new();
    key.insert("sessid".to_string(), av);

    let delete_item_input = DeleteItemInput {
        table_name: "session".to_string(),
        key: key,
        ..Default::default()
    };

    match dynamodb.delete_item(delete_item_input).sync() {
        Ok(_) => {
            debug!(applogger, "Deleted invalid session from ddb"; "sessid" => sessid);
        }
        Err(e) => {
            crit!(applogger, "Error in dynamodb"; "err" => e.to_string());
            panic!("Error in dynamodb: {}", e.to_string());
        }
    };
}

Lots, lots, lots more to do – but I’m having fun with this. It might go faster if I actually had a plan for what this web app actually does. I mean, I have an idea, but I haven’t mocked up a single page so I’m going slow and just enjoying hacking out Rust code as I go! Thanks for coming along with me!