I lead a project, which is using jenkins on top of thousands of generated views and is using jenkins views to see, and compare, various subsets. That all worked pretty fine untill some 3000 jobs and about 1000 of generated views. But since some 6000 jobs and 10 000 generated views it become useless.
So we searched for alternative. In reality there are two - Folders and Nested views. Where Folders are maintained, and widely used. Nested-views are orphan. Surprsingly, form all views (of our scope) nested views won.
So I adopted them, moved them to 21st century, and did few minor enhancements and realised, that nested-views were originally part of the jenkins itself, but were casuing issues for view extensions, so were extracted to plugin. And thus they have really interesting (and really misteriously working) codebase.
During all first testings it seemed that everything is workig moreover out of the box. What was the surprise, when we found that search sugestions works, search listing works but at the end, results links actually leads nowhere!
On above pictures you can see that both suggestion and listing are seamingly ok, when you check where the link folows, it is nonsense. See below:
The basic jenkins View class supports being searched through, so I immediately started to overwrite getSearchUrl later makeSearchIndex and was going deeper and deeper to unknown, until accepted the fate, that nothig I do, actually affects anything. Maybe it is somehow possible, but I have not found a way how to extend or modify basic jenkins search
On contrary, when reading powerfull https://github.com/jenkinsci/lucene-search-plugin it seemed that overwrite the search engine, should be very simple.
If you look into Lucene search's FreeTextSearchFactory.java and to corresponding FreeTextSearch.java impelmentation, You will see that you really have to only create jenkins extension point:
@Extension
public class NestedViewsSearchFactory extends SearchFactory {
@Override
public Search createFor(final SearchableModelObject owner) {
return new NestedViewsSearch();
}
}
Here hudson.Extensio's @Extension creates the extension as hudson.search.SearchFactory which literary mens you will overwrite the search engine. Once... The single and easy to implement createFor method then implements the real search:
package hudson.plugins.nested_view;
import hudson.search.*;
import org.kohsuke.stapler.Ancestor;
import org.kohsuke.stapler.QueryParameter;
import org.kohsuke.stapler.StaplerRequest;
import org.kohsuke.stapler.StaplerResponse;
import javax.servlet.ServletException;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.logging.Level;
import java.util.logging.Logger;
public class NestedViewsSearch extends Search {
private final static Logger LOGGER = Logger.getLogger(Search.class.getName());
public NestedViewsSearch() {
}
@Override
public void doIndex(StaplerRequest req, StaplerResponse rsp) throws IOException, ServletException {
String query = req.getParameter("q");
if (query != null) {
// hits = normalSearch(req, query);
// hits.addAll(manager.getHits(query, true));
}
//req.getView(this, "search-results.jelly").forward(req, rsp);
}
@Override
public SearchResult getSuggestions(final StaplerRequest req, @QueryParameter final String query) {
SearchResult suggestedItems = super.getSuggestions(req, query);
suggestedItems.add(new SuggestedItem(new SearchItem() {
@Override
public String getSearchName() {
return "aaaa";
}
@Override
public String getSearchUrl() {
return "bbbbb";
}
@Override
public SearchIndex getSearchIndex() {
return null;
}
}));
return suggestedItems;
}
}
Again, the hudson.search.Search interface is easy to implement and getSuggestions and getIndex are the warhorses who do the real thing. From here, you can do anything
See this commit
After above chnages, you can see that suggestions already works:
But the search itself not:
To display the results, we need to write our custom jelly file. Again, inspiration was found in lucene's search-results.jelly
To do so in my trivial usecase, I bound the jelly file to the src/main/java/hudson/plugins/nested_view/NestedViewsSearch.java - src/main/resources/hudson/plugins/nested_view/NestedViewsSearch/search-results.jelly:
<?jelly escape-by-default='false'?>
<j:jelly xmlns:j="jelly:core" xmlns:l="/lib/layout">
<j:set var="q" value="${request.getParameter('q')}"/>
<j:new var="h" className="hudson.Functions"/>
<!-- needed for printing title. -->
<l:layout title="${%Search for} '${q}'">
<l:side-panel>
<h2><img src="${rootURL}/images/16x16/help.png" /> Search help</h2>
<h3>${%Supported keywords with an example}</h3>
<p>${it.getSearchHelp()}</p>
</l:side-panel>
<l:main-panel>
<j:set var="hits" value="${it.hits}"/>
<h1>${%Search for} '${q}' returned: ${hits.size()}</h1>
<ol>
<j:forEach var="hit" items="${hits}">
<li>
<a href="${hit.getSearchUrl()}">${hit.getSearchName()}</a>
</li>
</j:forEach>
</ol>
</l:main-panel>
</l:layout>
</j:jelly>
This is verry dummy JSF dialect file, which in reality only iterates through single list of SearchItem (my List hits), but that should be all you need
The NestedViewsSearch.java was of course enhanced to implement all necessary methods called from jelly: getHits,and getSearchHelp (as this is really well done in Lucene search plugin)
package hudson.plugins.nested_view;
import hudson.search.Search;
import hudson.search.SearchIndex;
import hudson.search.SearchItem;
import hudson.search.SearchResult;
import hudson.search.SuggestedItem;
import org.kohsuke.stapler.QueryParameter;
import org.kohsuke.stapler.StaplerRequest;
import org.kohsuke.stapler.StaplerResponse;
import javax.servlet.RequestDispatcher;
import javax.servlet.ServletException;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.logging.Logger;
public class NestedViewsSearch extends Search {
private static class NestedViewsSearchResult implements SearchItem {
private final String searchName;
private final String searchUrl;
public String getSearchName() {
return searchName;
}
public String getSearchUrl() {
return searchUrl;
}
@Override
public SearchIndex getSearchIndex() {
return null;
}
public NestedViewsSearchResult(String searchName, String searchUrl) {
this.searchName = searchName;
this.searchUrl = searchUrl;
}
}
private final static Logger LOGGER = Logger.getLogger(Search.class.getName());
private List hits = new ArrayList<>();
public NestedViewsSearch() {
}
@Override
public void doIndex(StaplerRequest req, StaplerResponse rsp) throws IOException, ServletException {
String query = req.getParameter("q");
if (query != null) {
hits.add(new NestedViewsSearchResult("ccc", "../ddd"));
hits.add(new NestedViewsSearchResult("eee", "../fff"));
}
RequestDispatcher v = req.getView(this, "search-results.jelly");
v.forward(req, rsp);
}
@Override
public SearchResult getSuggestions(final StaplerRequest req, @QueryParameter final String query) {
SearchResult suggestedItems = super.getSuggestions(req, query);
suggestedItems.add(new SuggestedItem(new SearchItem() {
@Override
public String getSearchName() {
return "aaaa";
}
@Override
public String getSearchUrl() {
return "bbbbb";
}
@Override
public SearchIndex getSearchIndex() {
return null;
}
}));
return suggestedItems;
}
public String getSearchHelp() throws IOException {
return "nvp help";
}
public List getHits() {
return hits;
}
}
Note, that this class is re-initialised during each search, so be careful (but safe with) eg variable hits
the doIndex method was enhanced to actually do something. Add two dummy items:)
src/main/java/hudson/plugins/nested_view/NestedViewsSearch.java must have its jellys in src/main/resources/hudson/plugins/nested_view/NestedViewsSearch/
At the end you just show the new jelly page with freshly loaded results (hits):
RequestDispatcher v = req.getView(this, "search-results.jelly");
v.forward(req, rsp);
See this commit
Last step in this tutorial is to search in real data. Due to nature of my search - in views and jobs, the naive impelementation is very easy
private final List all = new ArrayList(1000);
public NestedViewsSearch() {
Jenkins j = Jenkins.get();
for (TopLevelItem ti : j.getItems()) {
if (ti instanceof FreeStyleProject) {
all.add(new NamableWithClass(ti, ti.getName(), ti.getName()));
}
}
addViewsRecursively(j.getViews(), "/");
}
Simply in constructor (remeber it is instance per search!) I iterate all jobs and all views, and read them to a bit weird structure with name and full path - to provide correct urls:
See this commit
The logic to fix urls is not nice:
private static class NamableWithClass {
private final String name;
private final String fullPath;
private Object item;
private NamableWithClass(Object item, String name, String fullPath) {
this.item = item;
this.name = name;
this.fullPath = fullPath;
}
public String getName() {
return name;
}
public String getFullPath() {
return fullPath;
}
public String getUsefulName() {
if (item instanceof FreeStyleProject) {
return name;
} else {
if (item instanceof NestedView) {
return fullPath + "/";
} else {
return fullPath;
}
}
}
public String getUrl() {
if (item instanceof FreeStyleProject) {
return "../job/" + name;
} else {
return "../" + getFullPath().replace("/", "/view/");
}
}
}
But major thing is, we are now in our own code. Whether to search by regex, contains, starts/ends, in name or in full paths, in views, jobs or in both, is just nice programming. My simple getSuggestions and doIndex now looks like below, and looking forward to tune them to theirs lmits:
@Override
public void doIndex(StaplerRequest req, StaplerResponse rsp) throws IOException, ServletException {
String query = req.getParameter("q");
if (query != null) {
for (NamableWithClass item : all) {
if (item.getFullPath().contains(query)) {
hits.add(new NestedViewsSearchResult(item.getUsefulName(), item.getUrl()));
}
}
}
Collections.sort(hits);
RequestDispatcher v = req.getView(this, "search-results.jelly");
v.forward(req, rsp);
}
@Override
public SearchResult getSuggestions(final StaplerRequest req, @QueryParameter final String query) {
SearchResult suggestedItems = super.getSuggestions(req, query);
for (NamableWithClass item : all) {
if (item.getFullPath().contains(query)) {
suggestedItems.add(new SuggestedItem(new NestedViewsSearchResult(item.getUsefulName(), item.getUrl())));
}
}
return suggestedItems;
}
hth, J.
The final implementation added some control switches and sorting of results. Note that suggestions do not have sorting.
See this commit
I'm not somehow proud on the implemetnation but it is fast enough on 10000s and 7000 jobs. Also the queries are quite intutitive and the results good
Note the final commitTo search from current position in jenkins may have some benefits and definitely needs some more programming. I had rather always search from top. Also the limitations and paging will come very handy