[Pkg-ofed-devel] New sources for RDMA upstream

[Ana Guerrero Lopez]

Hi Doug,

I'm sorry your email got rejected in pkg-ofed-devel at l.a.d.o, I had to make
posting allowed only to subscribers a few weeks ago due to spam :(
I whitelisted you and bounced your email to the list so the discussion
is properly archived.

On Wed, Sep 14, 2016 at 08:33:27PM -0400, Doug Ledford wrote:
> On 9/14/2016 7:37 PM, Jason Gunthorpe wrote:
> > On Thu, Sep 15, 2016 at 12:06:34AM +0200, Ana Guerrero Lopez wrote:
> >> While the Debian OFED team is mainly me,
>
> Just out of curiosity, if you are packaging up upstream, why do you call
> your group the OFED packaging group?  OFED is a specific group of
> packages, and not the upstream ones.

That's a fair question. This is purely because historical reasons, when
the team was created years ago, the team founders wanted to package the OFED
released (IIRC OFED 1.4 at the time) and they named the packaging group
in Alioth (Debian's forge) "pkg-ofed" https://alioth.debian.org/projects/pkg-ofed/

When I took over, making a new group didn't seem very worthwhile and it had
the potential to alienate existing people if I created a new group and moved
everything somewhere else. When I said OFED team I meant the team pkg-ofed
in Alioth.

> >> While I understand the rationale from upstream to make a single source
> >> repository and release it all at the same time, it doesn't really make
> >> Debian's packager life easier. I'd continue packaging things individually
> >> and that's likely to mean more work since I'd have to split the code.
> 
> I'm not really up on debian guidelines, but is this a requirement or a
> choice on your part?

There is not a requirement in Debian asking me to split the source and
there isn't anything in Debian policy against the new way you're proposing
to distribute the code.

When putting a package in Debian, it's necessary differenciate two things:

* _packaging_ the software: creating the debian/ directory and updating
when needed. This is not more complex that creating and updating a spec file.

* _maintaining_ the package in Debian, which means:

 - track carefully copyright and license changes and update debian/copyright
 - handle bugs from Debian users - that often are about upstream issues
 - handle bugs from other developers applying wide changes in the Debian
 archive (new compilers, toolchanis, reproducible builds efforts, people
 doing research in Debian and reporting things, etc)
 - investing time with upstream (i.e. these emails)
 - help handling security issues for stable releases
 - and others small things that require time and are not so evident.

Putting a package in Debian means accepting both tasks packaging and
maintaning the package.
The reply from Jason is focusing only in the packaging work and while
I agree having a single source makes packaging easier, it makes harder
maintaining the package in Debian as it needs more time for caring
for all the components and I'm not going to volunteer to do that.

With the sources split, I could care only about some parts of it, as
I do now that's what I said I would have to split to maintain it.

[...]
> >> Bundling it all together makes the packaging quite more complex -as your
> >> packaging shows- and the day to day maintenance is also more burdensome.
> >> This will raise the Debian contribution bar even more.
> 
> One moderately complex package vs. 20 simple packages.  There is
> slightly more required packaging acumen required to handle the package
> itself, but this is also offset by a reduction in complexity of the
> package dependency chain, so building becomes considerably simpler.
> There's no real way to quantify which is worse as it depends on what a
> packager is more familiar with, so this could be a net loss or win
> depending on the person in question.

The complexity of the packaging it's not the problem here for me, it can
be problem for other people joining the packaging since it makes harder
to understand how it all works.
I didn't really word very well in my first email in this point, so let
me try again re-wording the lines I quoted from my email:

Building it all together makes the packaging more complex, but more
importantly it makes the day to day maintenance also more burdensome since
any maintainer will have to care about all the components and this will raise
the Debian contribution bar even more.


> >> With the big tarballs every time there is a release, it will take a lot
> >> of more of time reviewing all the code, possible copyright changes,
> >> packaging updates...
> > 
> > I'm not sure I understand this. The number of patches will not change,
> > they will simply be in one place, not across > 18 different repos.
> 
> Indeed.  Code review should be identical in total amount, just different
> in that it happens on 1 package instead of 20.  And on any given update,
> the only thing that needs to be updated are the changes from the last
> tarball.  So if we release a new tarball that only has a one line bugfix
> to the built in cxgb4 provider, then that's the only thing that has to
> be reviewed.  If you aren't watching the git repos to know what the
> changes are, then unpacking both tarballs and doing a recursive diff
> between the trees is still better than doing a full review.

Because you're assuming I'd be reviewing the other 20 packages anyway
and it's not the case. When putting the full tarball of code in the Debian
archive I will have to look at everything anyway for things like copyright
changes, even if I'm not building the components.

 
> >> If there is a building problem, it also delays the full stack.
> > 
> > Yes, but that is counterbalanced by hopefully a more active upstream
> > looking at building issues and keeping track of toolchain progress.
> 
> It's also counterbalanced by the argument that this is a good thing.
> The RDMA stack is fairly tightly coupled.  If you have a build problem
> (which really should be highly unlikely anyway because we will be build
> testing before we release a new tarball), it indicates a problem that
> should be addressed and so this is an effective safety measure that
> helps to prevent a partial update that leaves things in an unknown state.

You can do a quite good job in preventing plenty of building problems, but
build problems are always going to happen, there will be people in Debian
building with new compilers and experimenting with the archive. And this is
a good thing.

[...]
> If you don't re-split things, I would think this would actually ease the
> maintenance requirements.  I know it will for our package people in
> Fedora and Red Hat (speaking as the main person who used to do that work).

As explained above, if you only talk about packaging, I agree, when talking
about maintaining in Debian, it's not the case.

I hope this makes my point clear.
Cheers,
Ana