development

Multi-module build based on sbt

March 15, 2024
Development, Documentation
scala, templates, development

import sbt.{Compile, Test, *} import Keys.{baseDirectory, libraryDependencies, *} // sbt.version = 1.6.2 ThisBuild / trackInternalDependencies := TrackLevel.TrackIfMissing lazy val welcome = taskKey[Unit]("welcome") val sparkVersion = "2.4.0-cdh6.2.1" val hiveVersion = "2.1.1-cdh6.2.1" lazy val commonSettings = Seq( //organization := "com.nnz", version := "0.1.0-SNAPSHOT", welcome := { println("Welcome !")}, scalaVersion := "2.11.12", javacOptions ++= Seq("-source", "15.0.10", "-target", "15.0.10"), libraryDependencies ++= sparkDependencies, resolvers ++= Seq("Cloudera Versions" at "https://repository.cloudera.com/artifactory/cloudera-repos/", ) ) lazy val root = (project in file(". ...

How to document your code?

July 12, 2019
Development, Tutorials
scala, templates, development

Comment documenter ? # Les mêmes principes et critères d’un bon code devraient s’appliquer à la documentation: Conventionnelle Simple Facile à comprendre En plus des critères d’un bon code, une bonne documentation devrait aussi être: Explicative (intention du code, règles métiers, clarification du code, mise en garde sur les conséquences d’une mauvaise utilisation, indications pour le testing) Non-redondante /** * Returns the temperature. */ int get_temperature(void) { return temperature; } Non-bruitée /** * Always returns true. ...

Hive

April 2, 2018
Utils
sql, hql, development

Snippets # -- set identifiers to none for the query below to work and -- set it back to column once it's done set hive.support.quoted.identifiers = none; HIVE 3 # BI Code typically use db.table - needs to change to db.table Default path : /warehouse/tablespace/external/hive/default.db/test_table Resources & Useful Links # ACID + HIVE