updated dependancies and fixed dependancies to point to databricks la…

…bs, and added quickstart tutorial to install the provider

README.md

@@ -1,51 +1,141 @@
 # Databricks Terraform Provider
 
 ## Table Of Contents
-  * [Getting Started](#getting-started)
-  * [Project Components](#project-components)
-    + [High Level Databricks Client CR[U]D](#high-level-databricks-client-cr-u-d)
-      - [Clusters](#clusters)
-      - [Libraries](#libraries)
-      - [Jobs API](#jobs-api)
-      - [Secrets](#secrets)
-        * [Secret Scope](#secret-scope)
-        * [Secret](#secret)
-        * [Secret ACL](#secret-acl)
-      - [Token API](#token-api)
-      - [Workspace (Notebooks) API](#workspace--notebooks--api)
-      - [SCIM API](#scim-api)
-        * [Users](#users)
-        * [Groups](#groups)
-        * [Group Member](#group-member)
-        * [Group entitlements](#group-entitlements)
-      - [MLFlow API](#mlflow-api)
-      - [Instance Profiles API (AWS Only)](#instance-profiles-api--aws-only-)
-      - [DBFS](#dbfs)
-    + [Databricks terraform provider resources](#databricks-terraform-provider-resources)
-  * [Testing](#testing)
-  * [Project Support](#project-support)
-  * [Building the Project](#building-the-project)
-  * [Deploying / Installing the Project](#deploying---installing-the-project)
-  * [Releasing the Project](#releasing-the-project)
-  * [Using the Project](#using-the-project)
-
+
+- [*Quickstart*: Building and Using the Provider](#-quickstart---building-and-using-the-provider)
+  * [Setup](#setup)
+  * [Downloading the source code and installing the artifact](#downloading-the-source-code-and-installing-the-artifact)
+  * [Basic Terraform example](#basic-terraform-example)
+- [Project Components](#project-components)
+  * [High Level Databricks Client CR[U]D](#high-level-databricks-client-cr-u-d)
+    + [Clusters](#clusters)
+    + [Libraries](#libraries)
+    + [Jobs API](#jobs-api)
+    + [Secrets](#secrets)
+      - [Secret Scope](#secret-scope)
+      - [Secret](#secret)
+      - [Secret ACL](#secret-acl)
+    + [Token API](#token-api)
+    + [Workspace (Notebooks) API](#workspace--notebooks--api)
+    + [SCIM API](#scim-api)
+      - [Users](#users)
+      - [Groups](#groups)
+      - [Group Member](#group-member)
+    + [Instance Profiles API (AWS Only)](#instance-profiles-api--aws-only-)
+    + [DBFS](#dbfs)
+  * [Databricks terraform provider resources](#databricks-terraform-provider-resources)
+- [Testing](#testing)
+- [Project Support](#project-support)
+- [Building the Project](#building-the-project)
+- [Deploying / Installing the Project](#deploying---installing-the-project)
+- [Releasing the Project](#releasing-the-project)
+- [Using the Project](#using-the-project)
+
+
 
-## Getting Started
+## *Quickstart*: Building and Using the Provider
+
+### Setup 
+
 Please note that there is a Makefile which contains all the commands you would need to run this project.
 
 This code base to contribute to requires the following software:
 
-* golang 1.13.X
+* [golang 1.13.X](https://golang.org/dl/)
+* [terraform v0.12.x](https://www.terraform.io/downloads.html)
+* make command
+
+To make sure everything is installed correctly please run the following commands:
+
+Testing go installation:
+```bash
+$ python --version 
+go version go1.13.3 darwin/amd64
+```
+
+Testing terraform installation:
+```bash
+$ terraform --version
+Terraform v0.12.19
+
+Your version of Terraform is out of date! The latest version
+is 0.12.24. You can update by downloading from https://www.terraform.io/downloads.html
+
+```
+
+Testing make installation:
+```bash
+$ make --version
+GNU Make 3.81
+Copyright (C) 2006  Free Software Foundation, Inc.
+This is free software; see the source for copying conditions.
+There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
+PARTICULAR PURPOSE.
+
+This program built for i386-apple-darwin11.3.0
+```
+
+### Downloading the source code and installing the artifact
+
+* After installing `golang`, `terraform`, and `make` you will now build the artifact.
+
+```bash
+$ go get -u github.com/databrickslabs/databricks-terraform && $ cd $GOPATH/src/github.com/databrickslabs/databricks-terraform 
+```
+
+* When you are in the root directory of the repository please run:
+
+```bash
+$ make build
+```
+
+* Locate your [terraform plugins directory](https://www.terraform.io/docs/extend/how-terraform-works.html#plugin-locations) 
+    or the root folder of your terraform code
 
-* python 3.6.X 
+* Copy the `terraform-provider-db` artifact to that terraform plugins locations
+
+```bash
+$ mkdir -p ~/.terraform.d/plugins/ && cp terraform-provider-db ~/.terraform.d/plugins/terraform-provider-db
+``` 
+
+Now your plugin for the Databricks Terraform provider is installed correctly. You can actually use the provider. 
+
+### Basic Terraform example
+
+Sample terraform code
+
+```hcl-terraform
+provider "db" {
+  host = "http://databrickshost.com"
+  token = "dapitokenhere"
+}
+
+// Creating a basic user
+resource "db_scim_user" "my-user" {
+  user_name = join("", ["test-user", "+",count.index,"@databricks.com"])
+  display_name = "Test User"
+}
+```
+
+Then run `terraform init` then `terraform apply` to apply the hcl code to your databricks workspace.
+
+Please refer to the detailed documentation provided in the html documentation for detailed use of the providers.
+
+Also refer to these [examples](examples/) for more scenarios. 
 
 ## Project Components
 
 ### High Level Databricks Client CR[U]D
 The client folder contains all the code for the golang sdk for the Databricks REST API. It is kept separate from the databricks terraform provider with its own unit/integration tests. The client so far supports the following:
 
 #### Clusters
-TODO!
+* [ ] Create cluster
+
+* [ ] Get cluster
+
+* [ ] Update cluster
+
+* [ ] Delete cluster
 
 #### Libraries
 * [ ] Create Library Installation
@@ -57,7 +147,13 @@ TODO!
 > Note: All library manipulation for clusters must be performed when the cluster is in a running state.
 
 #### Jobs API
-TODO!
+* [ ] Create job
+
+* [ ] Get job
+
+* [ ] Update job
+
+* [ ] Delete job
 
 Instance Pools
 * [x] Create Instance Pools
@@ -130,12 +226,6 @@ Note: For updating a user use Patch for entitlements, Put for everything else. W
 
 * [x] Delete Member
 
-##### Group entitlements
-TBD!
-
-#### MLFlow API
-TBD!
-
 #### Instance Profiles API (AWS Only)
 * [ ] Create IAM Instance Profile Link
 
@@ -161,15 +251,17 @@ TBD!
 * [x] Secret scopes
 * [x] Secrets
 * [x] Secret acls
-* [ ] Clusters
-* [ ] Cluster Policies
-* [ ] Entitlements
+* [ ] Jobs
+* [ ] Clusters (EC2 based example)
 * [ ] IAM Instance Profiles
+* [ ] Notebooks
+* [ ] DBFS
+* [ ] Libraries 
+* [ ] Cluster Policies (private preview)
+* [ ] Entitlements (private preview)
 
 ## Testing
 
-###TODOs
-
 * [ ] Integration tests should be run at a client level against both azure and aws to maintain sdk parity against both apis **(currently only on one cloud)**
 * [ ] Terraform acceptance tests should be run against both aws and azure to maintain parity of provider between both cloud services **(currently only on one cloud)**
 

client/client.go

@@ -124,6 +124,7 @@ func PerformQuery(option DBClientOption, method, path string, apiVersion string,
 			return nil, err
 		}
 		requestURL += "?" + params.Encode()
+		log.Println(string(requestURL))
 	} else {
 		if marshalJson {
 			bodyBytes, err := json.Marshal(data)

client/model/token.go

@@ -13,14 +13,3 @@ type TokenInfo struct {
 	ExpiryTime   int64  `json:"expiry_time,omitempty"`
 	Comment      string `json:"comment,omitempty"`
 }
-
-//
-//{
-//"token_value":"dapideadbeefdeadbeefdeadbeefdeadbeef",
-//"token_info": {
-//"token_id":"5715498424f15ee0213be729257b53fc35a47d5953e3bdfd8ed22a0b93b339f4",
-//"creation_time":1513120516294,
-//"expiry_time":1513120616294,
-//"comment":"this is an example token"
-//}
-//}

client/service/api.go

@@ -1,7 +1,7 @@
 package service
 
 import (
-	db "github.com/stikkireddy/databricks-tf-provider/client"
+	db "github.com/databrickslabs/databricks-terraform/client"
 )
 
 var scimHeaders = map[string]string{
@@ -55,6 +55,11 @@ func (c DBApiClient) Groups() GroupsAPI {
 	return groupsAPI.init(c)
 }
 
+func (c DBApiClient) Notebooks() NotebooksAPI {
+	var notebooksApi NotebooksAPI
+	return notebooksApi.init(c)
+}
+
 func (c DBApiClient) InstancePools() InstancePoolsAPI {
 	var instancePoolsAPI InstancePoolsAPI
 	return instancePoolsAPI.init(c)

client/service/clusters.go

@@ -3,7 +3,7 @@ package service
 import (
 	"encoding/json"
 	"errors"
-	"github.com/stikkireddy/databricks-tf-provider/client/model"
+	"github.com/databrickslabs/databricks-terraform/client/model"
 	"log"
 	"net/http"
 	"time"

client/service/clusters_integration_test.go

@@ -1,7 +1,7 @@
 package service
 
 import (
-	"github.com/stikkireddy/databricks-tf-provider/client/model"
+	"github.com/databrickslabs/databricks-terraform/client/model"
 	"github.com/stretchr/testify/assert"
 	"testing"
 )

client/service/groups.go

@@ -4,7 +4,7 @@ import (
 	"encoding/json"
 	"errors"
 	"fmt"
-	"github.com/stikkireddy/databricks-tf-provider/client/model"
+	"github.com/databrickslabs/databricks-terraform/client/model"
 	"net/http"
 )
 

client/service/instance_pools.go

@@ -2,7 +2,7 @@ package service
 
 import (
 	"encoding/json"
-	"github.com/stikkireddy/databricks-tf-provider/client/model"
+	"github.com/databrickslabs/databricks-terraform/client/model"
 	"log"
 	"net/http"
 )

client/service/instance_pools_integration_test.go

@@ -1,7 +1,7 @@
 package service
 
 import (
-	"github.com/stikkireddy/databricks-tf-provider/client/model"
+	"github.com/databrickslabs/databricks-terraform/client/model"
 	"github.com/stretchr/testify/assert"
 	"testing"
 )

client/service/instance_pools_test.go

@@ -1,7 +1,7 @@
 package service
 
 import (
-	"github.com/stikkireddy/databricks-tf-provider/client/model"
+	"github.com/databrickslabs/databricks-terraform/client/model"
 	"net/http"
 	"testing"
 )

client/service/main_test.go

@@ -4,7 +4,7 @@ import (
 	"encoding/json"
 	"github.com/joho/godotenv"
 	"github.com/r3labs/diff"
-	"github.com/stikkireddy/databricks-tf-provider/client"
+	"github.com/databrickslabs/databricks-terraform/client"
 	"github.com/stretchr/testify/assert"
 	"log"
 	"net/http"

client/service/secret_acls.go

@@ -2,7 +2,7 @@ package service
 
 import (
 	"encoding/json"
-	"github.com/stikkireddy/databricks-tf-provider/client/model"
+	"github.com/databrickslabs/databricks-terraform/client/model"
 	"net/http"
 )
 

client/service/secret_acls_test.go

@@ -1,7 +1,7 @@
 package service
 
 import (
-	"github.com/stikkireddy/databricks-tf-provider/client/model"
+	"github.com/databrickslabs/databricks-terraform/client/model"
 	"net/http"
 	"testing"
 )

client/service/secret_scopes.go

@@ -3,7 +3,7 @@ package service
 import (
 	"encoding/json"
 	"errors"
-	"github.com/stikkireddy/databricks-tf-provider/client/model"
+	"github.com/databrickslabs/databricks-terraform/client/model"
 	"net/http"
 )
 

client/service/secret_scopes_test.go

@@ -1,7 +1,7 @@
 package service
 
 import (
-	"github.com/stikkireddy/databricks-tf-provider/client/model"
+	"github.com/databrickslabs/databricks-terraform/client/model"
 	"net/http"
 	"testing"
 )

client/service/secrets.go

@@ -3,7 +3,7 @@ package service
 import (
 	"encoding/json"
 	"errors"
-	"github.com/stikkireddy/databricks-tf-provider/client/model"
+	"github.com/databrickslabs/databricks-terraform/client/model"
 	"net/http"
 )
 

client/service/secrets_scopes_acls_integration_test.go

@@ -1,7 +1,7 @@
 package service
 
 import (
-	"github.com/stikkireddy/databricks-tf-provider/client/model"
+	"github.com/databrickslabs/databricks-terraform/client/model"
 	"github.com/stretchr/testify/assert"
 	"testing"
 )

client/service/secrets_test.go

@@ -1,7 +1,7 @@
 package service
 
 import (
-	"github.com/stikkireddy/databricks-tf-provider/client/model"
+	"github.com/databrickslabs/databricks-terraform/client/model"
 	"net/http"
 	"testing"
 )