Add better support for generated resources

[joca-bt]

java_library.srcs supports .srcjar files to facilitate including generated sources. .properties files included in java_library.srcs are filtered and included in java_library.resources. Consider extending the current behaviour so that .properties files included in .srcjar files are also included in java_library.resources. This will facilitate the usage of generated resources.

For example, I have a genrule that generates .java and .properties files and I don't necessarily know all of them during the analysis phase for declaration purposes. For this reason, my rule instead generates a .srcjar file. However, the .properties files inside that .srcjar file are being skipped.

There are other alternative ways of going about it:

• There's already an internal resource_jars field that could perhaps be reused for this effect (at the moment it seems to always be forced as empty)? However, straight including .srcjar files in this field won't achieve the desired outcome as this can result in .java files appearing inside the final .jar file. Maybe somehow expose this field to users?

• Support something like .resourcejar files in java_library.resources. This would be a breaking change 😔.

[hvadehra]

You could also have your genrule generate two outputs, a foo.srcjar with just the .java sources, and a foo-props.jar with the .properties files.

genrule(
  name = "gen", 
  outs = ["foo.srcjar", "foo-props.jar"],
  cmd = 
    """
    # do whatever to generate the files
    zip -r $(location foo.srcjar) . -i *.java
    zip -r $(location foo-props.jar) . -i *.properties
    """, 
)

java_import(name = "props", jars = [":foo-props.jar"])

java_library(
  name = "lib",
  srcs = [":foo.srcjar"],
  runtime_deps = [":props"],
)

which should achieve what you want.

[joca-bt]

In this particular case, we need the properties to go inside the jar generated by java_library and not to be a runtime dependency, as that complicates distribution.

[hvadehra]

rules_java/java/java_single_jar.bzl

Line 76 in 3d75749

java_single_jar = rule(

can help with that.

[joca-bt]

Thanks for the suggestions. They work but I think we could make some improvements.

first

java_single_jar is missing from java/defs.bzl. Should it be added there?

second

in deps attribute of java_library rule <target that depends on java_single_jar target>:
java_single_jar rule '<java_single_jar target>' is misplaced here (expected cc_binary, cc_library, genrule, genproto, java_import, java_library, java_proto_library, java_lite_proto_library, proto_library, sh_binary, or sh_library)
and '<java_single_jar target>' does not have mandatory providers: 'CcInfo' or 'JavaInfo'.
Since this rule was created by the macro 'java_library', the error might have been caused by the macro implementation

Would it make sense to make java_single_jar compatible with java_library? Otherwise users need to java_import these targets which feels cumbersome. (Should I create a new issue for this one?)

[hvadehra]

java_single_jar was designed to create uber-jars without having to use a java_binary to create a _deploy.jar. I think making it return a JavaInfo may have run into some issues, @comius might know more about that.

For now, you could consider maintaining your own macro that declares an internal java_single_jar and a public java_import that wraps the former.

[joca-bt]

About this issue, which of these would be acceptable, if any?

• Supporting something like .resourcejar files in java_library.srcs
• Supporting something like .resourcejar files in java_library.resources

Both leveraging the internal resource_jars field for implementation.

I can work on a PR for any of these if that's an acceptable way of moving forward.

[hvadehra]

If the java_single_jar workaround isn't good enough, a PR to support resource jars SGTM. But I think it might be better to add a separate, explicit resource_jars attribute to bazel_java_library instead of more magic filtering. WDYT?

[joca-bt]

While I personally would prefer no magic filtering, I was trying to avoid adding a new attribute. srcs already supports .srcjars and there is no srcs_jar attribute, so there is precedence and it would maintain some sort of "consistency".

If the java_single_jar workaround isn't good enough

It is good enough for what we were trying to achieve, but I feel there could be improved support for resource jars like there is for src jars.

Closing this ticket is also an alternative if the maintainers aren't particularly happy with a solution at this time and would like to revisit in the future.

[hvadehra]

Well, we also have a resources parameter, so there is precedent in the other direction as well :)

The issues I have with the filtering, is:

1. we force using a specific/new extension, which means if you were somehow obtaining a .jar of resources from somewhere you couldn't just use it
2. it interacts badly with cases where you might have a ".resourcejar" that you want to include as is (without unpacking).

Which is why I think it's preferable to have separate attributes with simple/obvious semantics. Ideally, we'd have source_jars too and drop the filtering (but I can see why that's convenient for glob(["**"])s in OSS).

It is good enough for what we were trying to achieve, but I feel there could be improved support for resource jars like there is for src jars. Closing this ticket is also an alternative if the maintainers aren't particularly happy with a solution at this time and would like to revisit in the future.

Thanks for confirming. Lets keep this open for others to chime in if they end up needing something like this. If so, we can re-prioritize.