Chad's Logs

Python sets

chadgh — Sun, 25 Mar 2012 02:41:11 +0000

Just a quick programming tip/update.

I had to parse a file of 1.2 million lines. For those who don’t know, that is big. I needed to go through the file and keep track of the unique items in the file.

My initial implementation looked like this:

id = []
for item in items:
    if item.id in ids:
        continue
    else:
        ids.append(item.id)
    # do stuff

And that code ran for about 12 hours and was still not finished when I canceled it. I then learned a little more about sets in Python. By definition a set can’t have repeats, which makes searching through the structure for an item more efficient. The following solution finished in less than 5 minutes:

id = set()
for item in items:
    if item.id in ids:
        continue
    else:
        ids.add(item.id)
    # do stuff

I like sets. ;)

Be a man, use your…

chadgh — Fri, 24 Feb 2012 15:54:10 +0000

One of my favorite verses of scripture is 2 Nephi 2:26. This verse talks about the Savior’s gift to us and that, because of his actions, we are “free forever” and can choose between good and evil, “to act for themselves and not to be acted upon.”

I think a lot of times we forget (or maybe even don’t know) this vital truth about life. We are not rocks! We are not trees! We are not here to be acted upon. But what else would you say about someone who eats whatever is put in front of them (as I sit here eating a raspberry treat when I don’t even like raspberry, but I am eating it because it is here, within reach)? What would you say about the person who sits and watches hours of TV a night? Or the guy who gives into his base needs/desires/instincts and doesn’t attempt to control his appetites? I would say that they are being acted upon.

Note: I am writing this for myself, but I believe this applies to a lot of people.

Maggie thinks I am going through a mid-life crisis. However, I say that I am only now discovering that I need to act, I need to work towards being greater and better every day, week, month, and year.

Tangent: New Year’s resolutions are a joke. Why would you base your whole year’s success (or, more likely, failure) on some decisions you made at the beginning of the year? I mean, goals are great and all, but why not make them every month? Or better yet, once a month for big things, and then re-evaluate every week.

I recently read a very interesting and helpful article from what is rapidly becoming one of my most favorite web sites (artofmanliness.com), about what it means to grow up and be a man. I highly recommend everyone (not just men) read the article. I also highly recommend everyone (mostly men) visit frequently The Art of Manliness site to learn the lost art of becoming (and what it means to become) a man.

From the article there are two concepts addressed: freedom from and freedom to. Freedom from (aka negative freedom) can be summed up as “I am a slave to no man.” This is the ability that everyone has (as an adult) to choose between options. As an adult we are free to choose to do things that are not prohibited by law. We are free to go to parties, sleep in late, use drugs, and sleep around. As the image below illustrates, we are free to choose between options.

Freedom to (aka positive freedom) can be summed up as “I am my own master.” This is the ability that every adult has to master his appetites and to put up his own constraints and limits. A boy that wants to become a man places his own restrictions on his actions. The man sees things like the use of drugs and promiscuous sex as not uplifting or wholesome activities. The man puts up his own restrictions, knowing that he is improving his life and happiness, not by doing whatever he wishes, but by limiting himself and his appetites to improve the many aspects of his life.

Too often we think that being happy is the ability to choose between options. We experience the freedom to do what we wish after being restricted during childhood. We can stay up late, eat what we want when we want, and spend our time doing what we wish. However, growing up and becoming a real man/woman is realizing that being able to do those things doesn’t mean that we should. Growing up means choosing what is wise and right. I choose to put restrictions on myself and my actions because I know that doing those things is stupid, useless, or even damaging. I am not perfect and I do things all the time that are stupid, useless, and damaging. But I am trying to recognize those things each day, week, month, and year and I’m trying to be a better man as I grow.

In short, be a man, use your brain.

Bad Info: Potatoes are good

chadgh — Fri, 24 Jun 2011 22:43:11 +0000

In a recent study in the New England Journal of Medicine (June 23, 2011) there was a study published entitled Changes in Diet and Lifestyle and Long-Term Weight Gain in Women and Men. I first found about about this study from the following WSJ video produced the same day the journal issue was published.

When I heard the part about the potatoes it shocked me. I guess I should reveal my slight bias here, I grew up on a potato farm and my Dad still goes seed potatoes in Montana. But I was still shocked, since when is eating a vegetable bad for you? This just didn’t sit right.

I heard the study cited several times during the day including on a local radio talk show. Each time I heard it they pointed out that potatoes were found to contribute to more weight gain than anything else they looked at, just behind potato chips.

I decided to take a closer look at the study. Now I didn’t read the whole thing, but I did see this little nugget of information that some how no one else decided to share or at least was miss represented. Under the results section of the article there is a table of information (Table 2. Pooled, Multivariable-Adjusted Results for the Relationships between Changes in Dietary Habits and Weight Change.). In this table it shows the “Potatoes” category. According to this table over a 4 year period potatoes contribute on average about 1.28 lbs to a persons weight. But there was a little footnote to the potato:

For the categories … potatoes … subtypes were evaluated together in the full, multivariable-adjusted model in place of the overall food group.

Meaning that the two subtypes underneath potatoes (french fried and boiled, baked, mashed) were lumped in together. This confirmed my suspicion. People had been reporting that potatoes were bad for you contributing significantly to weight gain. But what this table tells me is that french fries not potatoes in general contribute significantly to weight gain. French fries indeed were found to contribute an average of (low end) 2.29 lbs over 4 years while other forms of potatoes (boiled, baked, and mashed) contributed 0.26 lbs.

People, please don’t misrepresent the facts. Potatoes are not bad for you. The way you eat them (what you put on them) might be, but the potato in and of itself it actually really good for you.

What I got from this study, be smart.

Google I/O 2011

chadgh — Mon, 23 May 2011 16:54:22 +0000

I attended Google I/O Developers Conference again this year (May 10-11). I have attended the last three conferences now and it keeps getting better and better (and I’m not referring to the free stuff I have received either). I have been able to learn which types of session I can get more out of and which things will make the best use of my time there.

Following is a detailed list of some of the sessions I attended along with some important bits of information I learned in the sessions.

Google Fusion Tables

Fusion tables provide a simple and easy way to store and visualize data.

Features (brief notes)

Easy to import data (KML, XSL, CVS, etc.)
Easily merge tables
Simple to create visualizations
Easily customize info windows
Simple to configure
Easy to share
APIs
You could build an application that uses FusionTables as a database through the RESTful APIs that are provided by sending SQL type queries. OAuth is used for authorization to a users account.
FusionTablesLayer
Can be used FusionTables with Google Maps JavaScript API to make faster maps with tons of markers. Images/markers are placed on the server side for quicker creation and dynamic updates.

mod_pagespeed

mod_pagespeed is an open source Apache module that speeds up web pages without changing what a developer has to do. Optimization of your web files (html, css, javascript, images) is done automatically (on the web server) before being sent to the users browser. Everything mod_pagespeed does is to limit both the number of files being sent back to the browser and the size of those files. Here is some more information too.
Also it is really simple and easy to set up (assuming your version of Apache is supported).

Some of what mod_pagespeed does before sending content to a users browser is:

optimizes (compresses) images
merges css files
minifies JavaScript
in-lines smaller resources
plus more

Google Storage for Developers

Google Storage for Developers provides the ability for developers to store data in the cloud (on Google servers). Currently there is a free trial available until December 2011. There is a browser client, command line utilities, and a REST API all provided to interact with the storage system.

Some other nice features mentioned include:

Built in load balancing and backups
OAuth2 and web browser loging (google account) for authentication
Tailored towards developers worried about scalability
resumable transfers (some quick notes):
- uses Google data resumable protocol
- uses an upload id for uploading
- resumable download uses GET range request

What can it be used for?

storage of applications
data sharing
static content hosting
storage for computation
backup and recovery

Google Chart Tools

Google Chart Tools provide a simple and easy way to visualize your data. With one data source many charts (visualizations) can be created (also works with FusionTables). It is programatically available through the Apps Scripts (ChartWrapper, ChartEditor). Also provides the ability to create controls for the data and visualizations and even dashboards.

Google Web Fonts

Web Fonts (in general)

The use of web fonts has the following benefits:

semantic
translatable
accessible
zoomable
efficient

Other key benefits are that the use of web fonts makes the web more searchable — Google loves web fonts. Web fonts are supported in 95% of all browsers and 100% of modern browsers. The standard way to use web fonts is with the use of the @font-face CSS rule. There are some major pitfalls of using the @font-face rule:

Font file conversion
hosting (mom and pop blogs ex)
css maintenance
browser quirks (Firefox)
licensing complexities

It is for these reasons that Google Web Fonts exists, to provide another simple and easy way for anyone to use web fonts to that they are used more. Google’s goal is to make web fonts ubiquitous.

Google Web Fonts

Google web fonts handles the browser quirks
no licensing worries
speed and reliability
font updates, optimizations and quality improvements
innovative font serving features

It is simple to use Google Web Fonts, just include the appropriate Google Web Font CSS file, then use the font in CSS (no need to specify the @font-face rule).

What’s next? (for Google Web Fonts)

More fonts (currently 175)
- aiming for 1000
- follow @googlefonts to news of new fonts added
faster fonts
- getting just the text of the font that you are going to use. Awesome! (text=”chad”)
- cached aggressively – fonts used on your site and on other sites, will be cached between the sites (on the browser).
in more places – TypeDNA – providing Google Web fonts through Adobe software

Speedy Maps

Some tips and pointers on making your map applications fast and responsive.

make each of your files as small as possible
require as few files as nessesary
Run your code through a compiler or a minifyer
- smaller
- simpler
- maintainable – you can have many comments and stuff in your code and it doesn’t get sent to the user. Closure compiler – provides syntax checking!
use multiple files for development, but then bundle the resources
use image sprits
consider embedding images in the html – when in production
- data url – ex: src=”data:images/png;base64,woeijasdf”
- reduces http requests
- simpler than spriting (conseptually)
- limited support in older browsers
- base64 increases image size by 1/3
bundle JavaScript files
only load essentials in head tag
keep common scripts seperate from infrequently-used scripts
bundle CSS files
avoid @import
place styles before scripts
Caching
- generally don’t case html
- aways cache JS and CSS, but use versioning to force them to be downloaded

Chrome Developer Tools

Chromes developer tool is just a web app (you can use inspect element on the developer tools!). Some of the features and tips listed below are in future releases of the developer tools.

Ctr-Shift-I to launch developer tools
(click on file name) revisions are kept of all the css changes that you make
free form style sheet editing
network tab – can record whatever is happening on in the browser
for JavaScript console functionality see commandline API (documented by firebug wiki)
A few examples:
- console.log
- dir() – gives you access to an object
- inspect()
- ESC – toggles on and off the console
- $0 refers to which ever element is selected in the dev tool
- copy() copies to your clipboard
Script Debugging
- Ctr-g – goto line
- hover over variable to see all its values at break
- break on subtree modifications – very handy if you don’t know where some code it happening.
- can set breaks for specific events
- can free form edit JavaScript
- unminifies minified code
- remote debugging –remote-debugging-port=1337
hit ? for keyboard shortcuts

HTML5 vs Native (Android)

One session had an interesting discussion on the pros and cons to building either an HTML5 app for mobile devices and the pros and cons to building native mobile apps (obviously focused on Android as the mobile platform).

HTML5 case

it is everywhere
natively working on most mobile devices (more coming each day)
in browser or embedded in a native app (web view)
flexible
- html – content – execs, users, customers
- javascript – logic – programmers
- css – style – designers
- graceful degradation – app doesn’t explode if something isn’t supported on a given device
- CSS – common.css, mobile.css, android.css (cascading!)
- reusing same backend, different frontends (desktop, mobile, car, tv, etc)
capable
- geolocation
- (multi) touch
- device orientation
- device api (catch different types of media/events)
- speech recognition
- semantic markup (meaning)
- rich graphics and canvas
- video / audio
- device cameras
- networking (ajax, cross-origin resource sharing, websockets)
- offline
open
- standards based
- searchable
- well understood tech
- well supported (libraries, debugging and dev tools, community)

Native case (Android)

anything running and built specifically for the app – android scripting layer using python (?)
gives you a richer experience
optimized for the system the user is running (device)
pros
- speed
- power
- control
- integration
“if you can’t improve using the native APIs, your not trying hard enough”
a standardized user experience
standards trail innovation
leverage the hardware that is available (cars, picture frames, tv, tablets, phones, etc)
native apps can interact with other apps
can be truely run in the background
alarms (cron job type apps)
full offline support (rich and already completely there on apps (coming to HTML5))
rich notifications
apps can be built on top of each other

Summary

HTML5 is here now, mobile, capable, open
Native is integrated, current, user experience

Be smart, use both, but at least use one.

A Very Morel Experience

chadgh — Sun, 24 Apr 2011 00:02:46 +0000

A few days ago I was walking through my yard taking note of all the dog turds that my neighbors dog had left for me to clean up when I stumbled upon something…um…well…freaky, frankly. My shoe had kicked something that had a little bit of resistance. When I looked down I saw at my feet something that looked like the image below.

a morel mushroom (morchella esculenta)

It was a morel mushroom (pronounced moral (at least that is what I was told))! Upon closer inspection I could tell it was a mushroom, however, I didn’t know what type it was or if it would kill me if I ate it. My first reaction was strange to me, I thought, “Yuck! I wonder if I can eat it.”

To the source of all knowledge, the Internet! I discovered the type of mushroom it was and that it was good to eat. I also found that there were some suggested precautions that should be taken. There were a few similar variety of mushrooms listed that were not safe to eat that I looked up to make sure I had the right mushroom type. My last and final safety measure, right after cooking it and right before eating it I called my Mom up and double checked with her. She said that her siblings used to pick and eat morel mushrooms a bit when they were kids. I took a bite. Mmm, I liked. I finished off the rest of the mushroom very quickly. It was good, I wish I could have eaten it with something else (like pasta or on a hamburger) but it was pretty good all by it self.

Fast forward to today. Right before I mowed the lawn I checked around the base of my (dead) apple tree where I had stumbled upon the first one, and found …

many much morel mushrooms, mmmm

Jackpot! Bingo! And whatever else one might say after finding such a treasure. I prepared a few for dinner tomorrow (cleaned and sliced) and fried the biggest two in butter. Mmm mmm good.

I hope I get a few more next year, but even if I don’t, I was glad for this very morel experience.