currently you need to call this with something like this:
pyspark --jars $HOME/dev/dirhash/target/sparkhacks-0.0.1-SNAPSHOT.jar dirhash.py /smartdata/ugfam/dev/test_data/raw/
But it would be better with something like this:
pyspark dirhash.py /smartdata/ugfam/dev/test_data/raw/
You can add the jars via python, as suggested here.
currently you need to call this with something like this:
pyspark --jars $HOME/dev/dirhash/target/sparkhacks-0.0.1-SNAPSHOT.jar dirhash.py /smartdata/ugfam/dev/test_data/raw/But it would be better with something like this:
You can add the jars via python, as suggested here.