Fix ImportFile() to send a multipart form request

[dheitman42]

Update ImportFile() to send a multipart form request, add missing "Name" parameter to ImportFileOptions.

[dheitman42]

I'll test this more thoroughly after the weekend.

[dheitman42]

Note this is intended to resolve #636

[svanharmelen]

No clue what you are trying to do here, but this doesn't look right to me. Can you please elaborate on this approach?

[dheitman42]

Sure, we're creating a multipart form, creating the file component with CreateFormFile(), copying the file contents to that part, then adding each of the fields from the ImportFileOptions struct as additional field data in the multipart form. That's the intent, anyway. I'll admit to being n00bish with golang, so maybe this isn't the proper way to do the struct-to-string-map conversion, but that's the basic idea.

[dheitman42]

Dunno if that answered your question, happy to clarify further if need be.

[zbindenren]

@svanharmelen this is how you do a multipart form upload: https://gist.github.com/mattetti/5914158/f4d1393d83ebedc682a3c8e7bdc6b49670083b84

In the api doc it is described as:

To upload a file from your file system, use the --form argument. This causes cURL to post data using the header Content-Type: multipart/form-data.

[zbindenren]

@dheitman42 I tried out your pull request and I got an internal server error. I tested our server successfully with the following code though:

func TestHelloWorld(t *testing.T) {
	extraParams := map[string]string{
		"namespace": "go",
		"path":      "test",
	}

	request, err := newfileUploadRequest("https://gitlab.example.com/api/v4/projects/import", extraParams, "file", "/tmp/file.tar.gz")
	require.NoError(t, err)
	request.Header.Set("Authorization", "Bearer "+gitlabTkn)
	client := &http.Client{}
	resp, err := client.Do(request)
        require.NoError(t, err)
}


func newfileUploadRequest(uri string, params map[string]string, paramName, path string) (*http.Request, error) {
	file, err := os.Open(path)
	if err != nil {
		return nil, err
	}
	fileContents, err := ioutil.ReadAll(file)
	if err != nil {
		return nil, err
	}
	fi, err := file.Stat()
	if err != nil {
		return nil, err
	}
	file.Close()

	body := new(bytes.Buffer)
	writer := multipart.NewWriter(body)
	part, err := writer.CreateFormFile(paramName, fi.Name())
	if err != nil {
		return nil, err
	}

	if _, err := part.Write(fileContents); err != nil {
		return nil, err
	}

	for key, val := range params {
		_ = writer.WriteField(key, val)
	}
	err = writer.Close()
	if err != nil {
		return nil, err
	}

	req, err := http.NewRequest("POST", uri, body)
	if err != nil {
		return nil, err
	}
	req.Header.Set("Content-Type", writer.FormDataContentType())

	return req, nil
}

[dheitman42]

Strange, I'm using the exact code from my MR in production, just verified it's working. I'll peek at your version when I have a spare moment (so, probably don't hold your breath :D )

[svanharmelen]

See my latest commit to this PR:

By using query.Values we are using the URL tags and are omitting any empty values. This seems to be inline with what the API expects. Could you both this this version @dheitman42 and @zbindenren?

[zbindenren]

@svanharmelen I am still getting a http 500 for my test:

	is, _, err := dst.ProjectImportExport.ImportFile(&gitlab.ImportFileOptions{
		Namespace: gitlab.String("linux/go"),
		Path:      gitlab.String("flash"),
		File:      gitlab.String("flash.tar.gz"),
	})
	if err != nil {
		fmt.Println("error", err)
		fmt.Println("body", err)
	}

go test -count=1 -v -run 'TestHelloWorld$' ./.
=== RUN   TestHelloWorld
error POST https://gitlab.pnet.ch/api/v4/projects/import: 500 failed to parse unknown error format
body POST https://gitlab.pnet.ch/api/v4/projects/import: 500 failed to parse unknown error format
    main_test.go:82: 
                Error Trace:    main_test.go:82
                Error:          Received unexpected error:
                                POST https://gitlab.pnet.ch/api/v4/projects/import: 500 failed to parse unknown error format
                Test:           TestHelloWorld
--- FAIL: TestHelloWorld (12.38s)
FAIL
FAIL    it.pnet.ch/golang/gitmig        12.388s
FAIL

[svanharmelen]

This should be fixed by PR #1318 which also adds some additional file upload improvements. Please open an issue if you have any issues with the new code. Thanks for the PR and the discussions!