SETA rewrite-Database Migration

Before I write note about GSoC, I really want to say this week is really messed up. My healthy status just like a roller coaster, my head is heavily sometimes and I got allergy(maybe). I can bare remember when I got skin allergy last time(maybe when I was 6 or 7 years old), and I don’t think see a doctor is a good choice because only thing they could do is give you some anti allergy, which can dizzy me whole day 😛 Whatever, let use flip to the next page.

Warning: Due to my poor sql knowledge, I actually have no ideal about how to describe my work of this weekend. I keep making mistake on very basic and simple thing and took me a lot of  time to figure it out. So, this blog may looks like a patchwork. I’m sorry about that. 😦

“If Time Can Roll Back, ……”

Alright, this title is all I think about during the migration work. “If time can roll back, I really should focus on my database class.”,”If time can roll back, I really should look more into it before I ask that dumb question.”, etc.

Anyway, first of all, after I can check the heroku log about SETA, I found the we have a error message like: Error R10 (Boot timeout) -> Web process failed to bind to $PORT within 60 seconds of launch And after I google it, I found this error is because I use ‘web’ dyno to boots my application and “web” type of application means that your app MUST listen some port. So, we capture $PORT from the environment variable  when boot the server. Now, ouija can ‘run’ on the heroku. But it’s far more than enough.

Problems keep showing up when I head to the database part. The first one is, I don’t know how to use it on heroku 😛 Everything look good to me except I found no data been actually wrote into the database! I spend a long time to find out why I can connect, and run sql command on database but no data will been wrote into it after that. Finally, I found I forget to ‘commit’ after I  execute sql command(Boom!)

Ok, we can import data into database now….oh wait, not yet. Error keeps showing up when I use failures.py to update jobtype. The reason is, in postgres, we can’t store a string array as varchar(which is what we do in mysql). The postgreSql as an Object-Relational database, has many data types which mysql hasn’t. For example, we store ‘jobtype'(e.g [‘android-4-3-armv7-api15’, ‘debug’, ‘crashtest-10’]) as varchar in mysql. But postgres has ‘array’ type and we could just define it as text array.  Furthermore, it tells me psycopg2.ProgrammingError: no results to fetch in this line. We can find the document said that A ProgrammingError is raised if the previous call to execute*() did not produce any result set or no call was issued yet, but it’s fine when we use it in Mysqldb 😛

After all, after resolve those problems mentioned above, we can visit ouija on heroku now (\o/)

Screen Shot 2016-05-15 at 8.39.18 PM

Advertisements

SETA rewrite-heroku deploy

The first step for SETA rewrite is about to deploy it on heroku platform. A little different place is we need to switch our database to Postgres-a heroku native database.

Why heroku

Heroku is a platform as a service (PaaS) that enables developers to build and run applications entirely in the cloud. When you need to deploy you code, the only thing need to do is link your github(let’s assume your code been stored in there) and push it to heroku. Done! Everything are set up then. The app deployment and code management are become much more easier and faster. And you can stretch the hardware configuration as you want. More information could been found in its home page.

I won’t spend too much time in this session because the only thing I did(as I said) is set up my local environment and **push**. The most sticky thing I meet in this step is I can’t use any add-one until valid my heroku account by credit card, but I don’t have it. We solve it at last by move its ownership  to the Mozilla crop. Thank you Armen 🙂

Database migration

PostgreSql, as I quote, is an object-relational database management system (ORDBMS) with an emphasis on extensibility and standards-compliance. As a database server, its primary function is to store data securely, supporting best practices, and to allow for retrieval at the request of other software applications. It can handle workloads ranging from small single-machine applications to large Internet-facing applications with many concurrent users.(I just copy from wikipedia, nevermind :P)

First thing we need to do is update our sql file so it can work on postgreSql. The postgres doesn’t use single quote for sql command or variable(column) name. For example, in MySql, we could create table like: ‘CREATE TABLE IF NOT EXISTS `tablename`’. But it must been updated as ‘CREATE TABLE IF NOT EXISTS tablename’. And postgres don’t have ‘datetime’ type for column, but we could use ‘timestamp‘(without time zone) instead. Ok, let’s see what we got now 😉

List of relations
Schema | Name | Type | Owner
——–+————+——-+———-
public | dailyjobs | table | mikeling
public | seta | table | mikeling
public | testjobs | table | mikeling
public | uniquejobs | table | mikeling
(4 rows)

All right, we should make ouija could connect to database and query from it now. This part of code is in updatedb.py. We should no longer use Mysqldb-python in there now because postgres is our main dish here, so we import psycopg2 instead. But it’s not enough for make it happened on postgres. The psycopg2 doesn’t support double quote in its .execute(). All right, let’s switch to ‘single quote’ now 😛

(SETA)MikeLings-MacBook-Pro-2:ouija mikeling$ python src/updatedb.py –delta 2
INFO:root:Downloader 1: a7783f2b4548 – 2016-05-07 02:11:30
INFO:root:uploaded 1/(13) results for rev: 27cfedea2aa7, branch: try, date: 2016-05-07 03:40:53
INFO:root:Downloader 1: 3966941f9dc3 – 2016-05-07 03:40:53

OK, we could deploy it on heroku and see if it can works on remote right now. But actually it’s not and I don’t know how to adjust it. Because I can’t access the log information(one disadvantage for lose your ownership). So, Some more works need to be done in next week after I can read log and configure app on heroku. BTW, I also fix one small issue for ouija.