Skip to content

Blog

  • in Python
  • 4 min read

NFLPool Progress 8/13/17

We’re a day away from launch. The NFL season starts in just over three weeks and players will need to submit their picks for NFLPool for the 2017 season. The majority of the picks won’t come until days before the season kickoff, but it would be nice to iron out some bugs… like the last one that might prevent me from launching for a few more days (but more on that later).

The last few days have been filled with big and little things. I had three big things to finish at the end of last week:

  • Get the email service working (welcome email and password resets)
  • Fix password resets (which I broke)
  • Add logging

Getting the email service working was harder than expected. To use a third party email service, like Amazon’s Simple Email Service, you have to update your website’s DNS records. My first problem was Amazon couldn’t create my account. After getting that resolved, I couldn’t get it to recognize my DNS changes no matter what I tried. I ended up going with Mailjet, which is working great. Just a bit more expensive, but it works.

Next was a day lost to password resets, and when I asked Kelly for help, needing a second set of eyes, it turns out the change I made to Pyramid’s routing broke it. I should have never, ever changed it. This was another case of me thinking I’m clever than I really am.

The good news with the routing, is that I think I might be starting to understand it, which is going to help with showing picks and standings once NFLPool has more than one season under its belt. More on that in a bit.

Lastly, I added logging. I followed the Python for Entrepreneurs class exactly, and now I’m using the logbook module successfully and am a happy Rollbar customer. (Thanks again, Talk Python!) Of course this means every time I break something while in development, I get an email from Rollbar that I have an error in my site, but it will be worth it.

I also finished a few smaller things, including importing the NFL schedule from MySportsFeeds and adding it to the installation process. This allowed me to write a check that if the first game has started to redirect the user to a page telling them it’s too late to submit picks. (Sorry, not sorry).

Once those things were complete, I moved on to the last big piece of functionality I needed to complete: showing the player the picks they just submitted. And things completely spiraled out of control after that.

Months ago Kelly sketched out what my data model should look like for storing a user’s picks.

I didn’t understand it then and it was the major reason development stalled this past spring. I thought I knew better and coded it the way that made sense to me. And of course, she was right. So this weekend was all about starting over. She helped me understand what I needed to do, and I had to add a number of new tables to the database and re-write dozens of methods to create the pics submission form and process it to store in the database. (But the good news is that if and when I convert NFLPool to MLBPool2, this fixes a major issue with MLBPool2 as users can change their picks at the All Star Break. I can now capture those changes and assign the points half value per the rules).

Then it was on to trying to show the user their picks. And that’s where we are – it’s hopelessly broken. I did figure out two cool things in Pyramid, which is auto-creating the pages for picks based on the season (this is related to routes from above) and after another quick query fix from Kelly, that works great. She was able to write a SQL query in SQL that does correctly pull all of a user’s picks, but I can’t seem to convert that a SQLAlchemy query that actually works. I’ve asked for help on Reddit and next up will be on StackOverflow. It’s all about the _and operator in SQLAlchemy which is way beyond my rudimentary understanding of Python and SQLAlchemy.

The last challenge is that every time I start working on something, whether it’s the admin installation process, user picks, or the user account page, I think of something cool that I want to add before I launch. I keep having to stop myself and instead I’ve started adding them to a to-do list in Wunderlist. Tom Clancy once wrote: If you don’t write it down, it never happened as Jack Ryan’s wife told him and it’s something that has always stuck with me. At some point I’ll convert them to Github issues and label them as bugs or feature enhancements and add them to milestones, but for now there is just too much to do.

And this is just to get the site launched and be able to have users register and submit their picks. I haven’t even started on the ability to compute a user’s score…. So much to do, so little time.

But I did change the homepage of NFLPool to reflect I’m launching tomorrow. It’s time.

  • in Python
  • 4 min read

NFLPool Progess 8/8/17 – Making a user’s picks work

I’m not even sure where to start from last update on Friday. The last 48 hours have been a roller coaster of up and downs of getting stuck and then figuring it out.

After completing the the functionality to create a new installation and add a new season to NFLPoo (or update to a new season) l, I needed to start working on letting a user make their picks for the new season.

First, I had to revisit my data model on where I store all the team information about each NFL team, including the division and conference each team lives in. During the new install process, I’ve added some code that now automatically assigns this. For the most part, this information is static. The NFL hasn’t changed what teams are in each division since the 90s and the only thing that may change at some point is if a team relocates and their abbreviation changes. For example, with the Chargers moving from San Diego to Los Angeles, they went from SDC to LAC. The Raiders are still a few years away from moving to Las Vegas, so I’m just going to leave that hard coded as is.

Now it was time to start coding the form that shows to users to make their picks. I couldn’t decide if this should live within its own controller or in the account controller. In the end, I’ve moved all of the picks to its own controller. But to do so, I need to re-watch all of the training videos on the routes, controllers and services to make this happen in Pyramid. It just wasn’t clicking for me. I was stuck Sunday, had a breakthrough, and by Monday night I was stuck again with a similar, but different, issue. I just couldn’t wrap my head around why and how you have to pass a dictionary from the controller and how it interacts with the data layer in the service. But finally, it clicked.

I finished wiring that up and now I needed to dig deep into Chameleon templates and Bootstrap forms with the Bootstrap CSS. Talk about getting stuck and being frustrated. The example used in the Python for Entrepreneurs course is pretty basic and I needed to pass a lot of values from the controller – one for each pick. I don’t know if there is a better way to do it, but I’ve figured it out by calling the service and a given method for each of the queries. For example, you have to pick which teams will finish 1st, 2nd or Last in a division (four teams in a division). This is one method per division. The method looks at the database and does a SELECT on the TeamInfo table WHERE the conference is equal to AFC and the division is equal to East and return the four teams in that conference and division. I got that working and was able to return two of the divisions. I didn’t want to do them all as I just needed to prototype that I could have two queries working and stick them in the form.

After I got that working, I needed to learn and understand how I could pass those attributes (the team name) inside of a dropdown box. Let me tell you – Google results for Chameleon templates, even when adding some of the TAL markup, are few and far between. You sure do see a lot of Powerpoint templates featuring chameleons in the search results for some reason.

There was so much swearing involved. I could iterate over the list, but then it would display four objects and not the team name. I kept at it, realizing I needed to fix the query, and now the team name did display. This was just inside a paragraph tag. I needed this to display inside a dropdown box and instead I made four dropdown boxes appear. (Progress? At least I was still iterating over the list.) And then I was stuck and asked my wife for help. She’s never used Python or Chameleon templates, but after about ten or fifteen minutes, she figured out the issue. To make matters worse, I was using the wrong Bootstrap button group, so we fixed that and she also figured out in Chameleon how to make the team_id display as the team_name and pass the team_id back to the form when submitting. I’m not going to even pretend I understand Chameleon templates and using more than these values – I understand you can do all kinds of Python code right in the template, but I don’t need it right now and it makes my head hurt.

That was huge. I then spent the last couple of hours before bed trying to figure out how to capture the user_id of the person currently logged in and making their picks, so I could submit that to the database so I know who actually made the picks. Lots of review of the BaseController from the training and I got it.

By that time it was past my bedtime, but I had the pick submission process working. I could grab the user_id of the person submitting the picks, the date and time the picks were submitted, and the picks themselves all grabbed from the form and passed to the database. The form also needs a lot of styling, but I’ll take usability first.

I’m so close to launching I can taste it – and with no time to spare. This week I need to finish coding the picks form and get all the choices in there, and then I need to add the password reset functionality to Accounts and add logging from Rollbar. It would be nice to log when a pick is submitted and get a notification, as well as any errors. Then it comes time to deployment, of which I have many thoughts, but that’s for another post.

  • in Python
  • 3 min read

Studying vs. Learning

I came across a thread on /r/LearnPython a few weeks ago that summed up my experience with Python. The thread was the typical “I’m learning Python, but I feel stuck and I’m not learning”. Someone replied with encouragement, and pointed out that studying Python is very different than learning Python and how it works.

I can relate. It’s been just over a year since I started down this path and almost two years (where does the time go!) since I decided I wanted to learn Python.

I started by buying a couple of books. That didn’t work for me at all – I don’t learn from reading and trying to apply the exercises. The Python for Everybody specialization at Coursera that I took last summer, with the video lectures and online exercises was the first time I felt I was actually learning. But in actuality, I was still studying. The same with the Python Jumpstart course. Having gone through Python for Entrepreneurs, and finally starting to apply what I’ve studied, I feel that I’m now learning.

Even though a large chunk of what I’m building in the NFLPool web app using Pyramid is copying and pasting from Python for Entrepreneurs, I’m starting to feel confident that the concepts are sticking.

As of Wednesday, I had a couple of the bigger components working. Users can visit the website and register, and the site recognizes when they’re logged in and out. I also created an admin panel that creates a new install and imports the NFL team information into one of the database table. This information is static and never needs to be updated. It uses a GET / POST / REDIRECT pattern and then redirects to the next page to insert all of the active NFL players for the new season.

Yesterday I sat down and needed to change this. Mr. Kennedy is always talking about the GET / POST / REDIRECT pattern , and though I thought I understood it, when I built the first one above, it took me a bit to get the redirect working. But I needed to change it so the next redirect would create a form to update the database for a new season and then redirect to import the active players.

I sat down and wrote the template and HTML code for the page, then wrote the controller for the GET / POST / REDIRECT routes, and then the service to interact with the database. I did this for the first time without referencing the training materials! The sense of accomplishment was huge. Not only was I writing code for the Pyramid web framework correctly, I was using requests and JSON to get the data MySportsFeed, manipulate it, and then put it in the database.

Of course, later yesterday, I wanted to write it so when you updated to a new season, it took a variable from a form rather than be hardcoded, and was stuck for two hours when it work. But it turned out it wasn’t the Python code – after my wife helped me for an hour, we realized it was the HTML form that had the error. So that was big, too – the Python code was right! It was just my lack of knowledge around HTML causing the latest headache.

Now I have two big things for the weekend:

  1. Re-write all of the requests so they’re not hardcoded and pass the season attribute from the database to the request so every year I don’t have to update the code.
  2. Right the controller and service for players to make their picks. This one is going to be tough – I’ll need to grab the user ID from their logged in session (which I’m not 100% sure how to do yet) and then grab a lot of stuff from the database and put it into a dropdown box for selection – which I also have no idea how to do. I feel ok that I know how to do the query, but putting the query results into a dropdown box is scary.

Either way, for the first time, I’m applying what I’ve studied – and I finally feel that after over a year I’m actually learning Python.

  • in Python
  • 6 min read

NFLPool Progress: 8/1/2017

I took the week off from work to see how much progress I could make on NFLPool this week (and to get some stuff done around the house, but really, for NFLPool). I told myself I’d blog my progress every day and here it is Wednesday already.

In my last blog post, I noted how I was going to have to move back to SQLite for the database. Over the weekend I ripped out most of the MongoDB code and started laying the groundwork for SQLite. The first thing I did Monday morning was going back through the SQLAlchemy chapter of Python for Entrepreneurs to learn how to properly set up classes in Pyramid using SQLAlchemy to create all the database tables. Back when I was building the first prototype for NFLPool, I had written a Python program that set up used the sqlite module and created all of the tables and then populated the database with the NFL teams information (Team ID, name of the team, city, etc.) and also populated the Active Players table with a list of all players in the NFL. This was all done using the SQL language – which I do not enjoy at all.

I won’t pretend that I still understand object oriented programming, especially the use of classes in Python. I’ve come a long way and I understand the concept of how a class creates an instance of the object, but putting it into practice is still a challenge. Using the Python for Entrepreneurs course examples, I was able to take my data model, write the SQLAlchemy classes, and get the database created.

One of the nice things about the course is that it does touch on how to create an index in a table and making foreign keys to join two tables together, which I’ll need. If I don’t ever migrate to MongoDB, I may want to just use MySQL, which technically, I believe, doesn’t support foreign keys. So I’m not sure how portable this code will be, but like everything else I’m doing, Future Paul can deal with that.

Monday afternoon was all about re-learning the Pyramid concepts of routes, abstracting the database in the viewmodel, and then writing the code in the controller. That took longer than I thought it would. The concepts are originally taught very early in the course, but then you go a long time without touching that part of the code. After all of the hands-on work, I think I’ve got a handle on it. I do like how Mr. Kennedy has you separate the viewmodel from the controller – he makes a point in the training you don’t have to do it – but if you don’t, I don’t see how troubleshooting your code could be easy.

That night, I took the kids to the community center where they swim while I sit and watch them in the viewing area with my laptop and use the free WiFi. I pulled my code down from Github and then swore in my head for an hour when Pyramid would error out and wouldn’t create the database. Which it just should – when Pyramid starts, it looks and if the database doesn’t exist, it looks at the classes I’ve written in SQLAlchemy in my /data directory and creates the database and those tables from the classes. But it wouldn’t. It wasn’t until I was back home later that night that I had an epiphany, created the /db directory manually where the database should live, re-started Pyramid, and voila, my database was there. Now I don’t have to worry about not being portable and can code both at home and on the go.

Tuesday was all about what happens when you first install and setup NFLPool. Since I’m using SQLite, I want to make it easy for me and other to set it up. The easiest thing to do with SQLite is when you need to make changes to the database during development to just delete it and have the app re-create it. Since I’m constantly developing it, I just wanted it easy to re-create and in the case my code is ever used by anyone (it’s on Github, licensed under the GPLv3), it’s better to do this now than try to add this functionality later. (Why GPLv3 and not MIT? My code is so bad being so new to programming, that I want any changes to the source available so I can look at it and learn. Though I’ll probably make it MIT at some point because no one is ever going to use my code, to be honest. Though I could digress on a business model I have brewing…)

Well, the first step in NFLPool is to set up the tables with the team information, which is static – it’s just the team name, the abbreviation, the city, and the division and conference they play in. So I thought I’d make an admin webpage where the admin goes, presses a button, and that database table is populated. It would then re-direct to the next page to create a new season. You enter “2017” for example, and it would reach out to MySportsFeeds and pull all active players in the NFL. Now you’re ready for the players in NFLPool to make their picks.

Sounds easy, and then the next thing I knew it was 7pm and I was asking my wife for help. Using the original prototype, which had working Python code to populate the team info table and the active players, I couldn’t get the team info to populate. I thought this tweet summed it up pretty well:

Turns out I had a return statement at the end of my loop that shouldn’t be there. Took her five minutes – sometimes you just need another set of eyes. The loop itself is terrible code, but it’s working.

The other problem I had was with the template and styling of the Admin page to create the first table. The Chameleon template engine kept crashing on me in my browser – again, she looked at it, pointed out I was missing two divs to close it, and that was working. Coding is fun, I keep telling myself.

After the table populates, which works now, I still don’t have the re-direct working correctly either. I can’t get it to re-direct to a new page – I’m pretty sure it’s how I have the POST and the routes set up in the admin controller. I can get it to redirect back to the same page though after it populates the first table. I need to keep pushing forward as vacation is almost half over, so there’s another thing on the TODO list for Future Paul.

Today is about doing some code cleanup. This includes adding an admin or settings table in the database to store the current season. From there, I can set up a baseurl function that connects to MySportsFeeds and using the season variable above to create the URL needed to get the JSON data needed for that entire season. This way, at the beginning of each season, you don’t have to go in and manually change all this code. I’m hopeful to get this done this morning, but will probably get stuck, but who knows. From there, I have two major pieces to get done:

  1. Start the code to allow players to make their season picks. From the Active Players table, query the database and get a list of all players in a certain position. Then create a form or Javascript to store those in a drop down box. I have no idea how to make a drop down box.

  2. Account creation / login / authentication. The course has a great chapter on this, I’m not too worried about it.

If I can get these done, then players can create their accounts and start making their picks (though I have to store those picks in the database, but I’m feeling confident). In theory, I could launch next week, which is tight. And then the hard part starts: writing all the code to calculate the scores for each player.

Until then, I’ll just keep telling myself this:

  • in Python
  • 2 min read

Back to SQLite for NFLPool

I’ve had to throw in the towel on MongoDB and move back to SQLite for the datastore. For now.

I was successful in being able to call an API, take the JSON object from the API, and store that JSON in MongoDB as a collection. But I what really wanted was to store that JSON object as an EmbeddedDocument within a collection.

My original goal was to stick JSON objects into MongoDB and then query against that. I envisioned two collections, one for Users (registration) and the second for the NFL data, called Stats. Stats would have looked like:

Stats

—Season

—2016 (and one document this fall for 2017, one per year moving forward)

—Player Picks (Each User’s picks for the 2016 season)

–Week (1 – 17, 17 different Documents, each embedded with:)

—Individual Player Stats

—Division Standings, etc

I read dozens of articles on StackOverflow, Reddit, and others. I tried to code it as an EmbeddedDocument, DynamicFields, and more. Everything ended up as an error. I needed to put the JSON object as an embedded document in each Week document above. But I couldn’t figure out.

I’m out of time. I’ve procrastinated enough and I really want to get a prototype and / or minimum viable product up in the next two weeks. There are two things that are a must:

  1. User registration and login system
  2. User pick submission

The first one shouldn’t be a problem as how to build a secure user registration system with passlib is included in the Python for Entrepreneurs course. The second requirements shouldn’t be that hard, but every time I think that, I’m proven wrong.

When I was using SQLite previously, I had all the code written to create the tables and do the initial import of the data needed. I’ll need to re-use some of that code and port the rest from the pure SQL statements I was using to SQLAlchemy.

I was also stuck on what the weekly results table should look like. My wife had some thoughts on it that thoroughly confused me, but when we chatted about it yesterday, neither of us could remember why she was recommending the design she did. I gave it some more thought and I think I have a database model that will work. I’ve written all the class files I need for SQLAlchemy and have taken the week off work to see how much progress I can make.

The big think I need to do is figure out the Javascript within Pyramid and the Chameleon template language to allow a user to make their picks. I have a table that stores all the active players in the NFL with columns for their first name, last name, player ID, team ID and position. I will need to create drop down boxes allowing them to choose, for example, which quarterback they think will lead the league in passing. That part of I have no idea how to do…. yet.

But one thing at a time.

  • in Python
  • 5 min read

NFLPool Prototyping with MongoDB

Yesterday was a good day.

With the static pages for nflpool.xyz complete, I started thinking about the dynamic pages. These are going to require access to the database and I’ll be using MongoDB. I had started the MongoDB course from Talk Python, but put that aside to go back to the Python for Entrepreneurs course to get the site up using Pyramid.

I took a step back and did some brainstorming about the data model I’ll need for the database. I grabbed a spare whiteboard and started scribbling.

nflpool whiteboard brainstorming

Using MongoDB requires you to shift your mental model from traditional SQL and joining tables as MongoDB doesn’t technically do joins. I needed to think about what a collection looks like and do I embed more documents within a collection or have multiple collections?

I went back through the MongoDB course chapter on Modeling and Document Design. Mr. Kennedy has you ask 6 questions when it comes to embed or not embed:

  1. Is the embedded data wanted 80% of the time?
  2. How often do you want the embedded data without the containing document?
  3. Is the embedded data a bounded set?
  4. Is that bound small?
  5. How varied are your queries?
  6. Is this an integration DB or an application DB?

The answers that I came up (that are hopefully correct):

  1. Yes
  2. Almost always
  3. I’m such a newbie I don’t even know what a bounded set is.
  4. I think so.
  5. Not very.
  6. This is an application database.

Keeping in mind that MongoDB has a 16MB limit, which sounds a lot smaller than it really is as I’m only dealing with text and not embedding images or anything like that, the answer is to embed everything.

The next step was to go back through the MySportsFeeds APIs and figure out which ones I’ll be using. In no particular order:

  • Cumulative Player Stats (for individual player picks, such as the passing yards leader)
  • Roster Players (used for the league players to pick who the individual player leaders are)
  • Playoff Team Standings (Used for wildcard playoff picks)
  • Division Team Standings (Used for which teams will finish 1st, 2nd and last in each division)
  • Conference Team Standings (Used for the team that will lead its conference in points for as well as some team data, such as the team name, abbreviation, etc.)

I may want the full game schedule at some point, but that’s a bigger challenge than I need to get into right now.

I ended up deciding that I need two collections within my database:

  • Users: this will store the registration information for each player in the league and be used for logging into the website
  • Seasons: Here I will embed all of the data for each season of play and have a document for 2016, 2017, etc. Within each year I’ll embed the league player’s picks and then a document for each of the APIs above that has 17 embeds – one for each week of the NFL season. One of my goals is that player can go back to 2016 and look at their progress for each week and see their point total versus the rest of the league. Getting ahead of myself, this will not be available in MLBPool2 (if and when I ever build that) as that will only be a real time look at your score and then show the final year results.

So it may look something like this, with two collections: Users and Seasons:

nflpooldatabase

–Users (embed a document for each player in the league in this collection)

–Seasons (Example: “2016” – and then embed the following in the 2016 document:)

  • 2016
    • player-picks
      • 1 document for each player’s picks
    • Week 1 through 17 (17 documents total) with the NFL stats for that week embedded here:
      • AFC East
      • 11 more documents for division picks like AFC East above
      • 6 documents for individual leaders
      • Tiebreaker
      • Documents for Points For, Wildcard, etc.
  • 2017
    • player-picks
    • Weeks 1-17 (One document per week)
      • NFL stats with all the embedded documents above

Now it was time to re-build the functions to go get the data from the MySportsFeeds API. I had this working in the last iteration of the app when I was using SQLite, but I’ve never used MongoDB before. Over my lunch hour, I successfully prototyped taking one query and putting it into my MongoDB running locally.

The feeling of euphoria in successfully using MongoDB was huge.

Last night after work, I took the next step. Keeping in mind the size limitation of MongoDB, I could take steps to filter the API calls, especially for cumulative stats. I only need a few key stats for a subset of all players in the NFL. For example, I just need passing yards for quarterbacks. The MySportsFeeds API provides a ton of stats for every player – such as fumbles, passes over 20 yards, QB Rating, completions, and defensive stats (even though they’re an offensive player).

Thankfully, Brad Barkhouse of MySportsFeeds is always available in the MSF Slack channel. I couldn’t figure out how to build a filter for just certain positions and a specific stat. (It turns out it’s just an & sign). So if I just want sacks for defensive players, it looks like this:

https://api.mysportsfeeds.com/v1.1/pull/nfl/2016-2017-regular/cumulative_player_stats.json?position=LB,SS,DT,DE,CB,FS,SS&playerstats=Sacks

So my task for the weekend is to figure out if I want to just embed a document for each individual category or one document with all of the cumulative stats and then just build queries for each category I care about (sacks, interceptions, passing yards, etc.)

I probably should just focus on the next phase of the Python for Entrepreneurs training though, and get the user login and authentication built and then go through the Albums part of the training, which I’ll mimic for the the league players to submit their picks. I’m running out of time as I really have only a few weeks before I need the player picks to be submitted before the start of the season.

Prioritizing is fun. But I’m so happy with some of the breakthroughs and progress and not trying to think of all the challenges ahead and just take it one step at a time.

  • in Python
  • 3 min read

Python for Entrepeneurs Progress

I sat down excited at dinner last night excited to share with my wife the two things I learned in my Talk Python course yesterday. The first was learning the basics of CSS, something I’ve avoided for years. I’m not going to even pretend I understand CSS, but it’s a base knowledge to work with and there is still a whole chapter of applied front-end frameworks, so I’m sure there will be more on CSS.

The second was a cache-busting technique, making it easy to both develop a website and see the changes right away without having to clear the browser cache and great for users that they’ll see the updates in production when it happens.

As I follow along in the Python for Entrepreneurs class, I’m trying something different this time. Rather than code along with the examples and do the examples as Mr. Kennedy does them, I’m trying to build nflpool.xyz using similar code as to what is is in the training. There has been a couple gotchas doing it this way, as you’ll start a chapter doing something one way and then learn a different and better way to do it. Overall, I kind of like doing it the way I’m doing it as the hands-on applications is one of the ways I learn best.

Unfortunately, the cache-busting code broke Pyramid and I got a myriad of errors in my Chameleon templates. I didn’t realize this until after I had started the routing section of the training and then lost an hour or two trying to trouble shoot the cache-busting. I finally gave up and ripped out cache-busting code from the templates and everything is working again. Well, working without the cache-busting code.

As I worked on the routing section, I’m not going to say I truly understand it yet, but it started to click for me why using a framework like Pyramid using Python makes sense. When I’ve mentioned to a couple of people that I’m going to use Python and Pyramid to build a site, I’m usually asked why I just wouldn’t use Javascript like everyone else these days. For me, focusing on one language at a time and not trying to learn too much is key. I’m pretty sure that I could do everything I want to do with nflpool in Javascript (including both the game calculations and website), but Python’s readability and reputation for being a good language to start with really appeals to me. So why wouldn’t I build it in Python? It gives me more hands-on experience with the language, which I need, and I can include the code needed for the scoring right into the application and don’t have to build two things – the game scoring and a website.

I still have a lot to get through this week. I need to finish the applied web development including forms (yay! Maybe I will have the ability to take user picks up by next month. Now, don’t get ahead of myself…); then front-end frameworks with CSS and Bootstrap; the biggest of them all – databases (more on that in a second); as well as account management; and finally, deployment. I skimmed the deployment chapter Sunday night – lots of good stuff (even if they use an Ubuntu VPS in the training) and I’m excited to give Ansible a chance.

One of the nice things about the trainings from Talk Python is that Michael Kennedy offers office hours for a Q&A section if there is something you’re stuck on. The next one is tomorrow, which is perfect. I’m going to see if I can solve my cache-busting problem, and if not, maybe ask for help. I also want to ask him his thoughts on using MongoDB instead of SQLite after taking his MongoDB training last week, while Python for Entrepreneurs uses SQLite.

I’m really enjoying the class and glad I’m making time for it. I don’t know if I’m going to have a skeleton up by the end of the weekend or not, but it’s coming along.

  • in Python
  • 4 min read

Stay on Target (or why my Python app still isn’t built)

One of the things I’m not doing well is focusing on one task at a time. As I continue to learn Python, every time I across a way to do something, I want to implement it right away without thinking ahead of how all the different things work together. Then I’ll get stuck, and frustrated, and my pace slows.

I need to find a task, stay on target, and just finish it, rather than jumping from feature to feature. With this in my mind, I’ve taken a step back to think about what needs to get done.

There are two major things that need to be built:

nflpool.xyz

Using Pyramid, I need to get the website up – even if this is just a skeleton. The Python for Entrepreneurs will get me there. I need to follow through and finish the course._

_

Major features for the website include:

  1. User creation / management: This includes creating an account, resetting passwords and login / logout. The course does an awesome job of how to properly hash and salt the passwords – I just watched and worked on this chapter yesterday.
  2. Yearly Player Picks: I will need to create a form for each player, after they have created an account, to submit their picks. This form will need to talk to the database to display the list of teams in each conference and the players available in each of the positions. I briefly looked at the Pyramid documentation Friday night and something like WTForms might work for this, but I really know nothing about it at this point. From there the player will need to hit submit, then review their picks or make changes, and then submit their picks which are stored in the database.
  3. Scoring: The last section of the website is the most important part to each player – how are their picks doing against everyone else? One of the reasons I’m using a database is that the cumulative player stats that MySportsFeeds provides are just that – cumulative through the season. There isn’t a way to just get the quarterback stats for week 5 of the 2016 season – so I need to store each weeks stats in the databases. This way a player can track their progress in nflpool through the year. Want to see where they stand right now? Check. Versus two weeks ago? Check. So the website will need to default to the latest week and then let the player choose the year and the week to see past history.

The only downside at this time for creating the website is if I want to use sqlite vs mongodb – I’d prefer to use mongodb as I’m stuck on how to create the individual player picks table and wanted to try it as a key / value store in a mongodb collection. The course is focused on SQLite with SQLAlachemy – something I’d like to learn but I think mongodb might also be easier for taking the JSON from MySportsFeeds and just sticking it right in the database.

nflpool app

The app has two major features that need to be completed:

  1. Import data via JSON from MySportsFeeds into the database: I had all of this done using SQLite. If I choose to switch databases, I’ll need to rewrite this this code.
  2. Scoring calculations: This isn’t done at all. This depends on the player picks table in the database, which is where I was stuck a few months ago when I took a break. I can’t figure out the data model for it no matter how many times my wife tries to explain it to me and I don’t know if I’m just being optimistic when I think a key value store in mongodb would work better. I’m going to give this a bit more thought and actually write out what the document would look like. This would probably be an embedded document in each user’s account.

Next Steps

With all that said, I think working on the website and getting a skeleton up is the best next step. If I can get the site up, then start to work on the submission form for the picks (which will require a bit of importing team data into a database, so I’ll have to make a decision there), I think I’ll feel a lot better. In a perfect world – and I know this isn’t going to happen in the next 30 – 45 days when I really need it, would be to have the submission form working prior to the 2017 season starting. Even if I have to still calculate the points weekly like I’ve been doing for the last two years, at least I’d have the picks in the database this time instead of having to work with the challenge of Google Sheets.

  • in Python
  • 3 min read

MongoDB for Python for Developers

I’m taking the latest training course that just launched a couple of weeks ago from Michael Kennedy at Talk Python: MongoDB for Python for Developers. This is my first exposure to NoSQL. Over the last year, I’ve searched Google a few different times trying to understand what NoSQL without any success – it always went over my head. Within ten minutes of starting this course, I think I might understand what a document database is.

I took a break from coding the nflpool app a few months ago after my wife gave me some feedback on how I was designing the data model for the SQLite database I was using. I was pretty frustrated, not with her, but just my lack of knowledge. I still hadn’t figured out how to import the individual player pick’s from the Google Sheet I was using, though I did find some open source code that did it perfectly. The challenge was if I changed the data model, the import functionality was going to change significantly and I couldn’t figure it out.

Here it is, early July, and I feel the panic of not having the app built for the upcoming NFL season for the second year in a row. That’s ok – it’s a marathon, not a sprint, to learn Python and build the app. Everyone needs a hobby.

I’m going to create a new branch in nflpool and see if I can use MongoDB instead of SQLite. I need to sit down this weekend and give some thought and sketch out the data model for the Users collection, but I think it could (should?) work better than what I was planning. It’s already obvious that importing the the NFL statistics from MySportsFeeds via JSON directly into MongoDB should be a slam dunk.

The challenge in switching is twofold: First, I’ll need to understand how that changes the Python For Entrepreneurs course – as I’m going to use Pyramid for the web framework, I’ll need to understand how those will work together. This is especially true for the user accounts and database sections of the course.

The second risk is by switching to MongoDB from a SQL language, there will be no help available from my wife. I might drive her crazy with my questions and the way I ask them, but she has a lot of knowledge of SQL and it might be even more of challenge doing the database on my own, in addition to the Python.

I’m enjoying the MongoDB for Python for Developers course. To be fair, it’s definitely over my head – I’m not a real developer nor do I have any kind of database experience or know any Javascript, so I’m taking it slow and in chunks. I’m not coding the examples as I follow along yet – I’m going to audit the whole course, give some thought to confirm this is what I want to do, and then I’ll go through it again. It’s probably in my best interest to finish Python for Entrepreneurs and get the Pyramid web app up and running. I do enjoy Mr. Kennedy’s courses – the way the courses are structured, how each lecture builds on the others and his delivery makes them worth the money.  Even for some of the topics where I don’t have the prerequisite knowledge I probably should, I find myself learning.  I’m on vacation next week and plan to spend a good chunk of time going through both the Python for Entrepreneurs course and the new MongoDB course.

Why I enjoy writing user help for GNOME

It’s been almost ten years since I started contributing to open source projects.  One of the big ways I’ve contributed in the past is writing user help.  Not knowing how to code then (and still really don’t know now, as hard as I try to learn Python), writing is something I enjoy and an area where I think I can make a difference.

There are a number of different places to apply a writing skill in open source.  You can write release notes, marketing copy, websites and the help documentation for an application.  Writing user help is one I enjoy.

I want to say that writing help is easy – but what’s easy for me, may not be for others.  Those who write an application might say it is easy for them – but as I’m learning, it’s not easy for me.

You can debate who might want to use a Linux desktop and not use Microsoft Windows or macOS.  To me, there are a few different use cases:

  • Developers.
  • Hobbyists.
  • Users in developing nations.

It’s these last two groups that I think having up to date user help is important. Using a Linux desktop, such as GNOME, can be a big change and paradigm shift for a user.  In developing nations, they may not be able to afford a Windows license or the applications they might want to run on Windows.  For example, Photoshop isn’t cheap – but GIMP is free.  If you’re switching to a new operating system, there may be things you don’t know how to do and if there isn’t user help available, how else are you going to learn?  Especially if you’re an area that might not have good internet access.

But having started to learn to code in the last year, I understand why developers don’t write help.  Even with my terrible skills at writing code, when I’m writing a function in Python, I’m not documenting my code as I should be, much less writing a document about how to use the finished application (if it ever gets finished).  You get in the zone and just write code and tell yourself you’ll get to it later.

But on the other hand, when I start an application and can’t figure out how to do something, my first step is to see if I can figure out how to do it myself.  I’ll check to see if there is help built into the application, if not, I’ll check the website.  Having come back to using GNOME a month ago, I was dismayed to find an application I was excited to use to not have either when I was trying to figure out how to use a feature.  (I won’t shame them publicly, and no, they aren’t an application created by GNOME, it’s an actively developed application available on Github).

Although I’ve been using macOS for the last few years, it’s not as if I stopped using open source.  I have open source applications running on my Macbook, a laptop running Fedora though I didn’t use it much, a server at home also running Fedora, and Digital Ocean droplets running CentOS and Fedora.  I strongly believe in open source and love that it’s powered both by people and companies building software in the open for anyone to use or modify.  And that last sentence is important – if anyone can modify it or make it better, why wouldn’t I help if I have the time and / or the skills?

So I did.  Jumping back in with both feet.  After my wife gave me some feedback on the app I’m building that I needed to re-architect a large part of it, I took a break from it for the last week and wrote user help for two apps in GNOME: Polari, an IRC client, and Recipes, a brand new application that does exactly what you think it does.  I’m even poking around the documentation for Builder, an IDE for building GNOME apps, and editing its developer documentation.  (I won’t even pretend I know how to write developer documentation).  It’s a nice change of pace to use a different part of my brain to write user help while my subconscious figures out how I’m going to fix the data model in my app.

Having been away from the GNOME community for a number of years, I’ve always said the one thing I missed about open source was the people – and it’s been neat to be welcomed back and see some of the same faces.  I love the collaboration.  Maybe someday after I finish my Python webapp I’ll learn GTK and make myself a desktop app out of it.  But let’s not get ahead of myself.