1. Efficient Tabling of Structured Data with Enhanced Hash-Consing
Neng-Fa Zhou
CUNY Brooklyn College & GC
Christian Theil Have
Roskilde University
Hash-Consing, Zhou & Have

2. Hash-Consing, Zhou & Have
The Problem
Current tabling systems suffer from an increase in complexity when dealing with structured data
B-Prolog (version 7.6 and older): O(N) space and O(N2) time
XSB: O(N2) in both time and space
Yap: O(N) space and O(N2) time
:-table is_list/1.
is_list([]).
is_list([_|L]):-is_list(L). ?- is_list([a1,a2,…,aN])
Hash-Consing, Zhou & Have

3. Hash-Consing, Zhou & Have
The Problem (Cont.)
This extra linear factor hampers application of tabled Prolog
:- table knapsack(+,+,max).
knapsack(_,0,0).
knapsack([_|L],K,V) :-
knapsack(L,K,V).
knapsack([F|L],K,V) :-
K1 is K-F,
K1 >= 0,
knapsack(L,K1,V1),
V is V1+1.
:-table hmm/2.
hmm(_,[]):-!.
hmm(S,[Ob|Y]) :-
msw(out(S),Ob),
msw(tr(S),Next),
hmm(Next,Y).
Hash-Consing, Zhou & Have

4. Hash-Consing, Zhou & Have
Current Solutions
Convert Prolog to Datalog by representing sequences as positioned facts (XSB)
Same representation has been used in Prism [Sato and Kameya 2008]
Auto-indexing of structured data through program transformation [Have and Christiansen 2012]
'C'(X,Y,Z) :- word(X,Y,Z).
Hash-Consing, Zhou & Have

5. Why the Extra Linear Factor?
is_list([1,2,3])
is_list([2,3])
is_list([3])
is_list([])
B-Prolog version 7.6 and older
The hash code is computed for each subgoal
XSB
No nodes are shared in the trie
Yap
Trie instructions are executed to return answers
Hash-Consing, Zhou & Have

6. Hash-Consing of Ground Terms
A technique originated in functional programming for sharing values that are structurally equal [Goto 1974; Appel and de Rezende Goncalves 2003]
Hash-consing in LP
[Neumerkel 1989]
[Nguyen and Demoen 2012]
[O’Keefe 2001]
Hash-Consing, Zhou & Have

7. Hash-Consing, Zhou & Have
Hash-Consing (Cont.)
Hashtable
is_list([1,2,3])
is_list([2,3])
is_list([3])
is_list([])
[1|.]
[2| . ]
[3| . ]
Hash-Consing, Zhou & Have

8. Hash-Consing alone IS NOT a Solution
Hash-consing can eliminate the extra linear factor in space
However, hash-consing does not change the time complexity since hash codes have to be computed
Hash-Consing, Zhou & Have

9. Hash-Consing, Zhou & Have
Input Sharing
Allows a tabled subgoal to share its ground structured arguments with its answers and its descendant subgoals.
Hash-Consing, Zhou & Have

10. Hash-Consing, Zhou & Have
Input Sharing (Cont.)
BEFORE
AFTER
Stack
Heap
Stack
Heap
[1,2,3]
[1,2,3]
Hashtable
Table area
Hashtable
Table area
[1|.]
[2| . ]
[3| . ]
Hash-Consing, Zhou & Have

11. Effects of Input Sharing
When a term is found to be in the table area, it’s unnecessary to copy the term again
When returning an answer that contains a ground structured term, it’s unnecessary to copy the term to the heap
BUT, hash codes still have to be computed
Hash-Consing, Zhou & Have

12. Hash-Consing, Zhou & Have
Hash Code Memoization
Hashtable
c1[1|.]
c2[2| . ]
c3[3| . ]
Hash-Consing, Zhou & Have

13. Enhanced Hash-Consing
Hash-consing + Input sharing + Hash code memoization
Enhanced hash-consing eliminates the extra linear factor in both time and space in dealing with ground structured data
It requires a hash table for ground structured terms and an extra word to store the hash code for each term
Hash-Consing, Zhou & Have

14. Hash-Consing, Zhou & Have
Evaluation
Enhanced hash-consing has been adopted in B-Prolog since version 7.7
The same hash-consing code is also used for findall/3, setof/3, and bagof/3.
Hash-Consing, Zhou & Have

15. Results on is_list([a1,a2,…, an]) Where ai’s are Random
Hash-Consing, Zhou & Have

16. Results on is_list([1,1,…,1])
Hash-Consing, Zhou & Have

17. Hash-Consing, Zhou & Have
The edit/3 Benchmark
:- table edit/3.
edit([],[],0).
edit([],[Y|Ys],Dist) :-
edit([],Ys,Dist1),
Dist is 1 + Dist1.
edit([X|Xs],[],Dist) :-
edit(Xs,[],Dist1),
edit([X|Xs],[Y|Ys],Dist) :-
edit([X|Xs],Ys,InsDist),
edit(Xs,[Y|Ys],DelDist),
edit(Xs,Ys,TailDist),
(X==Y ->
Dist = TailDist ;
sort([InsDist,DelDist,TailDist],[MinDist|_]),
Dist is 1 + MinDist ).
Hash-Consing, Zhou & Have

18. Results on edit([1,1,...,1],[1,1,...,1],D)
Hash-Consing, Zhou & Have

19. Results on edit(L1,L2,D) where L1 and L2 are Random
Hash-Consing, Zhou & Have

20. The Prism hmm Benchmark
:-table hmm/2.
hmm(_,[]):-!.
hmm(S,[Ob|Y]) :-
msw(out(S),Ob),
msw(tr(S),Next),
hmm(Next,Y).
Hash-Consing, Zhou & Have

21. Hash-Consing, Zhou & Have
Results on hmm
Hash-Consing, Zhou & Have

22. The create_list(N,L)Benchmark
:-table create_list/2.
create_list(N,L):-
between(1,N,I),
range(1,I,L).
?-create_list(10,L)
L=[1];
L=[1,2]; …
L=[1,2,…,10]
Hash-Consing, Zhou & Have

23. Results on create_list(N,L)
Hash-Consing, Zhou & Have

24. Hash-Consing, Zhou & Have
Conclusion
Hash-consing, enhanced with input sharing and hash code memoization, is effective for sharing structured data in tabled Prolog
With the extra linear factor gone, the scalability is greatly improved for applications that use ground structured data such as language parsing and bio-sequence analysis applications
Hash-Consing, Zhou & Have

25. Hash-Consing, Zhou & Have
Open Problems
Is it possible to have hash-consing for subsumption-based tabling?
Is it possible to have a trie-based data structure that can eliminate the extra linear factor?
How to deal with non-ground terms?
Hash-Consing, Zhou & Have