Statics are ignored

So I have setup the statics section of the fly.toml. But I can see the 404 from my go server picking up the request. So that makes me think the statics section isn’t being used.

[[statics]]
  guest_path = "/public"
  url_prefix = "/"

I am basically using the example go repo for fly. But I switched out to use Fiber. But I don’t have a Dockerfile or anything.

I am not using symlinks, that I know of. The only “building” is handled by the standard go buildpack from the go example.

Is there a step where I need to tell it to add the /public directory the way you do with a Dockerfile?

Per the go-example, / serves the template-based index.html.

Your statics section also says make the content of /public available at /.
Does your container contain a /public directory, for it to be found & served?

I don’t know how to see “inside” the VM, since there is no Dockerfile. But on disk, I have a /public folder with things like robots.txt and site.webmanifest. None of these are reachable, and instead routes me to the go server and I see the go servers 404 page.

So, my intent is to have the contents of /public served at the root.

I tried using other variations, like -

[[statics]]
  guest_path = "/public"
  url_prefix = "/public"

I did this just to see if / was a conflict. But those files not available inside my-domain.fly.dev/public either and I continued to get the go servers 404 page.

If I understand the [[statics]] concept correctly, these files should be served BEFORE my server receives the traffic. The fact my server gets the traffic means the layer that should intercept that isn’t working, or something is wrong with the configuration. But it’s fairly simple, so I can’t see anything wrong when looking at the docs.

I don’t know how to see “inside” the VM, since there is no Dockerfile.

Ah that’s true. This go-example doesn’t have a Dockerfile.
Strangely, there’s another go-example which does have a Dockerfile.
I don’t know which one Fly officially blesses, perhaps you could try the other one?

Anyway, as a general comment, you can use flyctl ssh to get into the VM and have a look.

If I understand the [[statics]] concept correctly, these files should be served BEFORE my server receives the traffic.

Yeah that’s right.

But on disk, I have a /public folder with things like robots.txt and site.webmanifest.

You mean your local disk or have you attached a volume to the app?

If the content you want to serve via [[statics]] isn’t a part of the build process / Dockerfile itself, it won’t work.

Very cool. I am not familiar with the structure inside the Firecracker VM. Is there a specific directory that contents are bound to when built? Or is it like docker where I need to get a PID and connect?

I will try and use a Dockerfile and see how that handles it. I will look at that example next and backport that into my app.

I am using embed for baking templates into the app when the Go app builds. So I am guessing it’s an issue with files not being moved over.

The [[statics]] section does not mention that.

Is there a specific directory that contents are bound to when built?

What you put on your final image via Dockerfile statements, only they’re eligible for being used in [[statics]].

Or is it like docker where I need to get a PID and connect?

For the SSH thing, the link will show you how.
In short, ssh establish and then ssh console.

I am using embed for baking templates into the app when the Go app builds. So I am guessing it’s an issue with files not being moved over.

You could also go through this thread and the link in it to see if that way of building your Go app suits you better.

It sounds like the go buildpack removes any folders/files that are not specifically marked as excluded from removal.

https://paketo.io/docs/howto/go/#prevent-source-files-from-being-deleted

The solution “appears” to be either having a project.toml file that specifies -

# project.toml
[ build ]
  [[ build.env ]]
    name="BP_KEEP_FILES"
    value="public/*"

Or to pass an additional argument on when building --env BP_KEEP_FILES="public/*"

The problem is, the project.toml does not seem to be used.

I tried using the [build.args] element within the fly.toml, but that does not seem to be respected. In the shell output I can not see the build command to validate.

[build.args]
  --env = "BP_KEEP_FILES=\"public/*\""

and

[build.args]
BP_KEEP_FILES = "public/*"

I can’t see a way to define ENV variable to be used on the builder in the fly.toml

I tried to add it as a secret to the builder. But it tells me it will be applied the next time it’s deployed. And you are not allowed to redeploy the builder.

Error Deploying over the remote builder is not allowed.

Are my choices creating a Docker file (go-repo example is 2 years old), or creating my own build pack, that I fully don’t understand?

I don’t believe you need a project.toml.
It can all be contained in the fly.toml itself, but with the right syntax, etc.
Here are two separate links to the doco, specifically the build section related bits.

So it’ll be something like:

[build]
  builder = "paketobuildpacks/builder:base"
  buildpacks = ["gcr.io/paketo-buildpacks/go"]
  [build.args]
    BP_KEEP_FILES = "public/*"

You may need a / or ./ in front of the public/* there.

In the shell output I can not see the build command to validate.

You could do $LOG_LEVEL=debug flyctl deploy ... to get more info on what’s going on.

I can’t see a way to define ENV variable to be used on the builder in the fly.toml

ENV variables are only available for use at run-time, and are setup via fly secrets.
For build-time arguments, the two links above will help.

And you are not allowed to redeploy the builder.

Quoting an answer from an employee:

Are my choices creating a Docker file (go-repo example is 2 years old), or creating my own build pack, that I fully don’t understand?

It appears that using the Dockerfile is what you have to do if you want to serve static files (that you can make available in your final Docker image).

Creating your own buildpack is not necessary, I think.
Just need to pass the build args right.
I hope the above links will help get you going :slight_smile:

Appologies, I thought I had listed that I had tried that also. In the Pako example they show some ENV with ./ and other without. I tried both. Here is the actual current config I am using -

[build]
  builder = "paketobuildpacks/builder:base"
  buildpacks = ["gcr.io/paketo-buildpacks/go"]

  [build.args]
    BP_KEEP_FILES = "./public/*"

My point on ENV is that build.args is an argument to the builder. But the command that is passed to pack is --env BP_KEEP_FILES="./public/*"

In other words, it’s unclear to me that --env makes it as part of the command. That is why I was asking if there was a equivalent to the project.toml build.env.

https://paketo.io/docs/howto/go/#prevent-source-files-from-being-deleted

Also, I was under the understanding that it uses an existing deployed builder, and does not create a new builder each time. But treating it like a long-lived CI/CD VM.

Appreciate the help. Let me see if I can learn any more setting the $LOG_LEVEL

Quick comment before I read the whole thing.
It’s just LOG_LEVEL by the way, I meant to denote the terminal prompt by $.
So LOG_LEVEL=DEBUG flyctl deploy ... is the command.

I think I’m may not have properly understood what you mean here. With that in mind…
You shouldn’t use --env (as indicated in the pako guide) in the build.args.

For fly, flyctl docs page dictates what’s available.
That is, --env is to pass build args. The alternative to this is the build section in fly.toml.

Also, I was under the understanding that it uses an existing deployed builder, and does not create a new builder each time. But treating it like a long-lived CI/CD VM.

That is correct. I meant that destroy builder text as a response to ‘not allowed to redeploy the builder’.

Yeah, I figured that out. =)

So I ran the deployment with debug. I don’t see the “pack build” command run anywhere. But I do see this -

Provided Environment Variables
  BP_KEEP_FILES=./public/*

So it does appear they are treating it as ENV variables. I am assuming to the build environment.

The next line talks about using a cache volume.

Using build cache volume pack-cache-showdown-labs_cache-d600ad7458d1.build

I wonder if nothing is being applied because setting an ENV does not cache bust?

Don’t know about that I’m afraid.
Have to call the class teacher (Fly folks) lol.

By the way, did you try with a Dockerfile?

Not yet. I have not done a lot with Docker. The example from 2 years ago looks like it may not be the best to base it on. Trying to find a better boilerplate to work off of.

That is my next plan.

It’s kind of the same thing baked into the flyctl built-ins.
So worth a try.

https://github.com/superfly/flyctl/blob/master/internal/build/imgsrc/builtins/defaultbuiltins.go

So I found that repo really helpful. I was able to quickly create a Dockerfile and build it locally and prove it can serve the app with no issues. But I am failing to modify it to copy the public folder over.

Does anyone have more docker experience than me that can point out what I am doing wrong? This example is largely the one from the buildpack.

FROM golang:1.17-alpine as builder
WORKDIR /go/src/app
COPY . .
RUN go mod download
RUN CGO_ENABLED=0 GOOS=linux go build -v -o app
FROM alpine:latest
COPY --from=builder /go/src/app/app /goapp/app
COPY --from=builder /go/src/app/public /goapp/public
WORKDIR /goapp
COPY . /throwaway
RUN cp -r /throwaway/resources ./resources || echo "No resources to copy"
RUN cp -r /throwaway/public ./public || echo "No public to copy"
RUN rm -rf /throwaway
RUN apk --no-cache add ca-certificates
ENV PORT=8080
EXPOSE 8080
CMD ["/goapp/app"]

When I navigate to the container, I expect the /public director to be within /goapp/public but it’s not present.

That looks like it should be working ok. One quick fix, move this line up right underneath FROM alpine:latest:

RUN apk --no-cache add ca-certificates

That’ll make your Docker build faster. :slight_smile:

Try adding ls -la /goapp/public right before CMD and see what the docker build prints out?

I got this working with the following changes -

FROM golang:1.17-alpine as builder
WORKDIR /go/src/app
COPY . .
RUN go mod download
RUN CGO_ENABLED=0 GOOS=linux go build -v -o app
FROM alpine:latest
RUN apk --no-cache add ca-certificates
COPY --from=builder /go/src/app/app /goapp/app
COPY --from=builder /go/src/app/public /goapp/public
WORKDIR /goapp
COPY . /throwaway
RUN cp -r /throwaway/resources ./resources || echo "No resources to copy"
RUN cp -r /throwaway/public ./public || echo "No public to copy"
RUN rm -rf /throwaway
ENV PORT=8080
EXPOSE 8080
CMD ["/goapp/app"]

And in my fly.toml I had to adjust the guest_path to align with the goapp folder.

[[statics]]
  guest_path = "/goapp/public"
  url_prefix = "/"

I was also able to remove the build.args from fly.toml. These never seemed to work, and removing this did not impact the build.

  [build.args]
    BP_KEEP_FILES = "./goapp/public/*"

Thank you @FrequentFlyer & @kurt