Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple fixes and improvements #1

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

felipecsl
Copy link

  • Fix gemspec files and dependency on extlib

This fixes 3 issues when installing/using the gem:

1 Missing files referenced in gemspec:

wukong at /Users/felipe/.rvm/gems/ruby-1.9.3-p194/bundler/gems/wukong-d656650bdae9 did not have a valid gemspec.
This prevents bundler from installing bins or native extensions, but that may not affect its functionality.
The validation message from Rubygems was:
  ["examples/server_logs/breadcrumb_edges.rb", "examples/server_logs/histograms.rb", "examples/server_logs/page_counts.pig", "notes/away/icss/icss_specification.textile", "notes/away/swineherd/swineherd-old-readme.textile", "notes/away/wukong-fs/wukong-fs-old-readme.textile", "notes/away/wukong-transform/wukong-processor-old-readme.textile", "spec/data/a_atsigns_b.tsv", "spec/data/a_follows_b.tsv", "spec/data/tweet.tsv", "spec/data/twitter_user.tsv"] are not files

2 Missing dependency on extlib:

➜  wukong git:(master) ✗ bundle exec irb

1.9.3-p194 :001 > require 'wukong'

LoadError: cannot load such file -- extlib/blank

3 Missing dep on right_aws:

/Users/felipe/.rvm/gems/ruby-1.9.3-p194/bundler/gems/wukong-f25cbb17dbd1/lib/wukong/script/emr_command.rb:1:in `require': cannot load such file -- right_aws (LoadError)
    from /Users/felipe/.rvm/gems/ruby-1.9.3-p194/bundler/gems/wukong-f25cbb17dbd1/lib/wukong/script/emr_command.rb:1:in `<top (required)>'
    from /Users/felipe/.rvm/gems/ruby-1.9.3-p194/bundler/gems/wukong-f25cbb17dbd1/lib/wukong/script.rb:154:in `require'
    from /Users/felipe/.rvm/gems/ruby-1.9.3-p194/bundler/gems/wukong-f25cbb17dbd1/lib/wukong/script.rb:154:in `run'
    from /Users/felipe/.rvm/gems/ruby-1.9.3-p194/bundler/gems/wukong-f25cbb17dbd1/lib/wukong.rb:16:in `run'

@felipecsl
Copy link
Author

Also found and fixed a problem with emr_extra_args where it was exploding here:

/Users/felipe/Data/Projects/wukong/lib/wukong/script/emr_command.rb:76:in `execute_emr_runner': can't convert String into Array (TypeError)
    from /Users/felipe/Data/Projects/wukong/lib/wukong/script/emr_command.rb:32:in `execute_emr_workflow'
    from /Users/felipe/Data/Projects/wukong/lib/wukong/script.rb:155:in `run'
    from /Users/felipe/Data/Projects/wukong/lib/wukong.rb:16:in `run'
    from visits_runner.rb:7:in `<main>'

@felipecsl
Copy link
Author

Added Settings config option to allow specifying which AWS region to run the job via emr_region option

@mrflip
Copy link
Owner

mrflip commented Oct 4, 2012

sorry for neglecting this -- we've moved the primary wukong branch to infochimps-labs/wukong, so I totally missed this. If I haven't merged this in by later this week please poke us: http://github.com/infochimps-labs/wukong/issues.

@felipecsl
Copy link
Author

Alright @mrflip thanks :)

@felipecsl
Copy link
Author

Any update on this, @mrflip ?

@felipecsl
Copy link
Author

Is this repository still maintained, or we should be using the infochimps-labs version?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants